ghost-ray

AI Computer Vision and Solana to Incentivize Marine Debris Collection

Comment

Inspiration

Ghost nets and marine debris are silent killers in our oceans. Current collection efforts lack a transparent, immediate incentive for local collectors. We wanted to build a system where the "proof" of work isn't a paper receipt, but a digital verification that triggers an instant reward.

What it does

  1. Uses a custom YOLO model and Gemini 2.5 Flash to identify debris and estimate its mass via a live camera feed.
  2. Once verified, the backend signs a Solana transaction to dispatch a SOL reward from a central "UN Treasury" wallet directly to the collector.
  3. All metadata (location, mass, material) is pushed to IPFS (Pinata) to create a permanent, decentralized record of the collection event.

How we built it

  1. Backend: Python FastAPI server handling the logic.
  2. Computer Vision: YOLO for object detection and Google Gemini for high-level environmental analysis. Gemini estimates mass by analyzing the object's pixel volume and material density relative to known environmental reference points in the camera frame. It then applies the physical formula mass = density x volume, where it approximates the volume based on visual dimensions and assigns a density constant based on the identified type of debris.
  3. Blockchain: Solana (Devnet) via the Shyft API for transaction drafting and signing.
  4. Frontend: Next.js with a clean interface designed for global deployment.

Challenges we ran into

While the original vision included Compressed NFTs, I realized that implementing Metaplex Bubblegum via Python is a complex challenge that requires a deep understanding of Merkle Tree proofs and serialization.

Accomplishments that we're proud of

I decided to prioritize a working, reliable payout system over a buggy NFT implementation. I prioritized the SOL transfer and IPFS metadata storage over the NFT gallery to guarantee that the collector always receives their incentive without being blocked by network congestion.

What we learned

I gained hands-on experience with the Solana transaction lifecycle—from drafting a transaction via the Shyft API to manually signing it using the solders library and broadcasting it to the Devnet RPC. I learned that while Solana is incredibly fast and cheap, the developer experience is heavily centered around Rust, Anchor, and TypeScript. Coming from a Python background to integrate AI, I had to learn how to bridge these two worlds.

What's next for ghost-ray

  1. Full Lifecycle cNFTs: Use cNFTs to track debris status from Collected (Minted) → Verified (Updated) → Recycled (Burned).
  2. Hybrid Architecture: Build a dedicated Node.js microservice to handle complex cNFT transactions more reliably.
  3. On-Chain Logic: Migrate the payout system to an Anchor (Rust) smart contract to make the platform fully decentralized.

Built With

Share this project:

Updates