Deqo

A Decentralized and Secure Marketplace for Compute

Comment

Inspiration 🌟

In today's technological landscape, the demand for computational power has reached unprecedented levels, particularly in AI and machine learning. While cloud providers like AWS and Google Cloud offer robust solutions, their services often come with significant costs that can be prohibitive for many users. Meanwhile, an enormous amount of computing power lies dormant in personal computers, university clusters, and corporate infrastructure during off-hours. This untapped potential represents not just wasted resources, but also missed opportunities for both providers and potential users. We can make the market for computing power more efficient by using those resources.

What it does 🔧

Deqo (Decentralized Compute Marketplace) is like Airbnb for computing power. Deqo lets people and organizations rent out their unused computing power to others who need it (like AI researchers, data scientists, or small businesses) and earn money in return.

However, what makes Deqo special is its focus on security and ease of use: Blockchain technology ensures all computations are verified and secure.

Key features include:

  • Blockchain-based verification for secure computation integrity
  • AI-powered resource matching and smart scheduling system
  • Secure containerization with AWS ECR and real-time encrypted monitoring
  • CLI interface with Supabase RLS authentication for automated execution

How we built it 🛠️

Our system architecture is divided into four key components, each working together to create a secure and efficient compute marketplace:

  1. Frontend Platform
    • Next.js 13 + TypeScript responsive interface
    • Real-time dashboard for job monitoring and earnings
    • Interactive resource scheduling and availability
    • AI-powered job matching based on hardware specs
  2. Backend Infrastructure
    • FastAPI on AWS EC2 with auto-scaling and load balancing
    • Secure Docker image management via AWS ECR
    • Encrypted data transfer through AWS S3
    • Real-time state management with Supabase
  3. Security Architecture
    • Blockchain-based computation verification
    • End-to-end encryption (AES-GCM) with asymmetric authentication
    • Secure code execution via Docker isolation
    • Multi-layer computation proof system
  4. CLI Integration
    • Automated task management with Supabase RLS auth
    • AI-powered requirement verification and job execution
    • Offline-capable scheduling with blockchain reporting

From the perspective of the buyer: They upload a docker image to the portal, and once someone accepts the job and runs the docker, they can download the returned output files from their code.

From the perspective of the lender: They log on to the portal, find jobs that suit their needs, and run them. Using the CLI we built, they can just input a job ID, and the docker will run autonomously. They can also autorun the CLI, where it will find run jobs at preset times suitable to the lender, making passive revenue completely autonomously.

Challenges we ran into 🚧

The development process presented several significant technical challenges:

  1. Security Implementation: Creating a robust security system that could safely execute arbitrary code while protecting provider resources required extensive testing and validation.
  2. Resource Management: Developing an efficient algorithm for matching compute jobs with available resources while considering factors like pricing, availability, and hardware requirements.
  3. Verification System: Implementing a cryptographic verification system that could validate computation results without introducing significant overhead.
  4. State Management: Building a reliable system for tracking job states across a distributed network while maintaining consistency and fault tolerance.

Accomplishments that we're proud of 🏆

Our team successfully delivered a production-ready platform within 36 hours that includes:

  • A fully functional marketplace for compute resources
  • Secure container-based job execution system
  • Real-time monitoring and analytics dashboard
  • Automated payment and verification systems
  • Comprehensive provider controls for resource sharing

What we learned 📚

This project provided invaluable experience in distributed systems architecture, including container orchestration and cryptographic verification systems. We gained deep insights into real-time state management across distributed environments, ultimately learning how to build and scale secure marketplace platforms.

What's next for Deqo 🚀

Our next major milestone is implementing true distributed computing capabilities across our network. This means that a single computation task could be automatically split and processed across multiple provider devices simultaneously, significantly increasing processing power and efficiency.

Our vision is to create a powerful distributed supercomputer from idle devices around the world, making massive computational power accessible to everyone while ensuring providers are fairly compensated for their contributions.

Built With

Share this project:

Updates