Groktext

Start the conversation while it still matters.

Comment

Inspiration

We love Community Notes. They provide an invaluable layer of context over X posts — crowdsourcing disagreement until some overwhelming consensus is reached. Yet, that crowdsourced mechanism suffers from a critical latency gap. Currently, a note is typically attached only after a post has achieved viral critical mass. We were interested in how we could begin the conversation surrounding a potentially inflammatory or controversial post early, so more of the community is a part of the conversation.

What it does

We built this project to solve that timing problem. Our goal is to decouple consensus from virality, ensuring that high-quality context is available before misinformation (or ill-contextualized, emotional content) spreads, rather than serving as a retrospective correction.

How we built it

  1. Ingestion & Analysis: We ingest the realtime X filtered posts (limiting to conversational engagement like replies and quotes — the closest we could get to a general engagement firehose) via the API. Each engagement is paired with its parent (or child) post that sparked the conversation. This brings us to the first part of our ingestion:
    • The Filter: We calculate summary statistics of the engagements within every interval (we chose 10 minutes) and find the moving average of engagement for some parent post within that interval. Each tweet first under two parallel processes:We select the parents that got engagement greater than 1 stdev from that avg to move on to the next step.
    • Velocity Tracking: We calculate a moving average of quotes and replies to establish a baseline for viral engagement, and select the parent posts that received engagement greater than 1 stdev from that average. These are potentially controversial posts sparking conversation rather than simply likes.

  2. Narrowing: We then isolate the high-risk content by identifying posts that meet both criteria:

    • Marked as inflammatory by Grok (sent via a request w/ picture and text content to 4.1-fast).
    • Exceeding the 10-minute moving average for engagement.

  3. Candidate Generation: Selected posts and their metadata are routed to a dedicated backend service that generates multiple candidate Community Notes.

  4. The AI Jury: A simulated jury evaluates these candidates using In-Context Learning (referencing historical successful notes from the X dataset) to predict helpfulness.

  5. Selection & Rendering: The candidate with the highest bridging score is selected, attached to the post, and rendered on the user client.

Challenges we ran into

We burned through our X API credits in around ten minutes, which meant we could not run this service live, and instead relied on the cached data for mock ups and data exploration. Although this seems quick, this gave us >77k pieces of conversation to work with. From there, we selected ~700 parents driving those conversations. After sending a request to Grok, we were left with around 100 potentially inflammatory or dangerous posts (those that might benefit from extra context).

The Grok API is extremely slow at generation of requests, when the model has access to web search, which means we could only do very limited prompt tuning.

Accomplishments that we're proud of

Our end to end pipeline actually ended up working with real X data — live! In addition, when we read through some of the Grok Notes, they ended up reading just as we would expect Community Notes to read.

What we learned

We learned that it's actually quite difficult to distill high quality user reviews for Community Notes into few shot in context learning to use for synthetic juries with Grok. This is for a few reasons, the first is that Grok often has difficulty with embracing the cognitive dissonance associated with partisan fact-checkers. The second reason is that Grok is extremely sensitive to the system prompt, and even slight tweaks could result in the consensus of a tribe dropping by 50%. This means that quite a lot of tuning is needed in practice for juries via ICL to work.

What's next for Groktext

We'd love to work with the X and xAI team to implement an alpha or beta version of Grok Text! There are many improvements to make, such as adaptive clustering (re-run the SVD/Clustering step automatically every 24 hours on the new data collected by the system). Adversial attack countermeasures, such as defending against brigaders who purposefully upvote a false community note. Even incorporating advertisting friendliness metrics, such as finding, which tweets are both becoming viral and have high consensus (lucrative for advertisers).

Share this project:

Updates