Neo Cloud - The CapEx

Multi-million dollar investment before you charge $2/hour GPU rent

The Neo-Cloud tech stack is primary designed around Nvidia echo system, other than few exceptions. The key challenge of such design was that you are starting from a very high price point where most of money is already spent acquiring BOM of Nvidia hardware, Network switches, Storage, Fiber cables, CPU servers, which left very little money, if any, to spend on software license eg. VMWare, Nutanix

Here is basic BOM ( Bill of Materials) for setting up an Nvidia SuperPod (32 nodes), which is really small (300kW - 400kW)

This is very high level price and you have not put a single dollar into ground yet, ie. this does not include cost of Data center Co-lo ( which could be atleast 200K - 300K ). Now you would need a team to setup this infrastructure, till Elon gets us robots

As you can see, the price escalates very quickly as you scale the cluster, even by adding just a few more SuperPODs. Employee costs do not scale linearly, but hardware costs and power consumption do.

Now in order to create a business of scale, not a basement cluster, you need more than 1 such SuperPOD. Neo-Cloud H100 clusters vary widely in scale. Providers like CoreWeave operate very large fleets around 250,000 GPUs globally across dozens of data centers with dedicated facilities housing more than 3,500 H100s1 in a single AI supercomputer deployment.

Other Neo-Clouds such as Crusoe Cloud provide multi-GPU clusters capable of hosting hundreds of H100s for sustained model training, and platforms like Lambda Labs explicitly support multi-node configurations from tens up to thousands of GPUs as customer demand scales

For simple math, lets take 128 node cluster or 1024 GPUs, considering the Dell base systems of power rating of 10kW2 , it would be around 1 MW facility with CapEx of $16M * 4 = $64 Million.

Total One time Capex = $64M

So far we have put $64M, which is mostly Nvidia Revenue and some went to other OEM/ODMs. Cool thing being a startup is that, investor don’t ask for their money for very long time.

So, now lets the the math for OpEx and Revenue.

The OpEx

There primary 2 operational cost

1. Co-location Fee - Which often combined with Electricity cost

   In Northern Virginia, one of the most active data center markets in the world, wholesale colocation space can cost $120–$190 per kW per month, meaning a 1 MW rack hall could cost $120K–$190K3 each month just to reserve power capacity. On top of that, commercial electricity rates in Virginia run around 10–11¢ per kWh, further increasing monthly operating costs for high-intensity GPU clusters.4

2. Employee Cost - Which remains relatively flat as the cluster scales. However, to run even a simple cloud operation (baremetal), following critical roles need to be filled and some could be offshored to reduce the cost

   1. Data Center Technician - Onsite

   2. Third-party contractor for Ad-hoc repair activity

   3. 2 X SRE (Could be offshored)

   4. 2 X Network Engineers (Could be offshored)

In total, we can clearly see the year 1, total investment stands around ~70M with around $4M - $5M of recurring expenses. Now lets Dive into the business model of revenue generation.

The Revenue

Neo-Cloud revenue today spans a mix of product streams: bare metal GPU rentals, virtualized GPU instances, and higher-level API/managed offerings.5

Finding earning and revenue of any Neo-Cloud is pretty simple, all you need 2 variables, the Price of GPU/Hr and total number of GPU in their inventory.

So revenue (ARR)of our Neo-Cloud setup with 1024, with 70% utilization would be anywhere between $14M - $26M.

However, revenue and profit picture is not as simple as it seems here. Its hard to be always 100% utilization, GPU price volatility and also other operational costs ( Sales, HR, Office and other salaries) play as well.

Profits, ROI and Margins

If we run simple math to find the Net cash flow with 75% Utilization

ARR with 100% Utilization

The table below summarizes how utilization directly impacts payback time and annual return for a 1,024-GPU H100 Neo-Cloud deployment.

Moreover, contrary to believe that GPU rent prices are dropping is not necessary true across all the market. If I compare H100 price GPU rent prices between Sep 2024 and Feb 2024, I see a price appreciation rather than Price drop.

The whole narrative that Neo Clouds are money losing business and need constant upgrade may not be accurate. If done right, Neo Cloud are pretty decent business for being backbone for AI infrastructure and meet the customer’s insatiable demand for GPUs.

The reason for H100 price appreciate could be following

• More LLM adoption across Enterprise eating up all Hyperscaler inventory

• Lack of Power availability for new Blackwell based chip causing demand back-flow to H100

• More VC money pouring into building various AI products driving up the Neo-Cloud GPU demand

Next Blog, I will take a deep dive into the Linux based Tech Stack which powering all these Neo-Cloud AI Infrastructure.

The whole AI boom and bust cycle is incomplete without mentioning or understanding Neo Cloud economics, which often get overshadowed by hyperscaler billion dollar investment news cycles and discussions around gigawatt scale data center facilities.

In this blog, I will take a look at Neo Clouds, first derivative picks and shovels play for AI, from a technical standpoint and also at how the business model has evolved over the years. Even though I have been working in the Neo Cloud space for over five years now, I did not fully understand it until I listened to the famous Odd Lots podcast, where the CoreWeave CEO breaks down the bigger picture of AI Neo Clouds and how they transitioned from a crypto miner to an AI cloud provider.

For those outside the core technology space, transitioning from a crypto miner to a GPU cloud might seem natural. But under the hood, the technology stack is very different, to the point where you almost have to redesign everything, from data center architecture to software tooling.

Before I dig into the Neo Cloud tech stack, let me first break down the different classes of Neo Clouds in the market and their business models. I will be focusing only on GPU focused clouds that primarily target AI and LLM development shops.

This space is constantly evolving, and many players frequently change their business models by moving up or down the technology stack.

What Neo-Clouds actually are

• Neo-Clouds are a new class of infrastructure providers built primarily around GPUs and AI workloads.

• They sit between hyperscalers and traditional hosting providers, both in terms of scale and complexity.

• They are not trying to build a general purpose cloud with hundreds of managed services.

• Their core value is reliable access to GPUs, fast networking, and just enough software to run AI workloads.

• Most Neo-Clouds exist because it’s hard to certain kind of GPU’s with Hyperscaler as they have mostly pre-allocated them to their premium clients or for their internal applications.

• Neo-Clouds focus on utilization and efficiency rather than feature breadth.

• Neo-Cloud pricing are very volatile with cut throat competition and survival of the fittest.

What Neo-Clouds are not

• They are not hyperscalers and do not compete directly with AWS, GCP, or Azure.

• They are not model companies and do not own or train foundation models as their primary business.

• They are not API-only AI platforms that abstract away infrastructure completely.

• They are not trying to hide the infrastructure from users; in most cases, the infrastructure is the product.

• They do not promise magic or simplicity at all costs. They sell compute, and everything else in the stack exists to make that compute usable and rentable.

Neo-Cloud Categories

Neo-Cloud - Marketplace & Hybrid Resellers

Neo-Cloud - Pure Play

You might notice that I have used Mid-Market as a broad blanket category for many providers. The simple reason is that the Neo Cloud business model can be very complicated. In many cases, these providers are selling each other’s excess supply and frequently switching roles between being a market maker and simply reselling someone else’s capacity to make a margin.

In the table below, I have provided some basic criteria for grouping and categorizing these players.

Categories

Marketplace

These are players who do not own any infrastructure but instead provide a platform where other Neo Cloud providers can sell their excess capacity. Shadeform is a market leader in many ways, considering its broad market access, snappy developer friendly UI, and fast onboarding with quick launch capabilities.

Hybrid

This category gets a bit complicated, as many players frequently change their business models. A good example here is Hydra Host, where they may operate their own colocation facilities while also collaborating with other data center providers to sell cloud GPU instances as if they were their own.
The key difference between a marketplace and a reseller is that, in the case of a reseller, the end customer may not know whose infrastructure their workload is actually running on. In many cases, these hybrid cloud providers also offer L1 customer support to help customers directly.

Own and Operate

These are classic Neo Cloud providers who actually own and operate their own fleet of data centers and GPUs. This is a capital intensive endeavor, and not many companies can sustain or survive at this level. In this category, the clear market leader is CoreWeave, with customers like Meta, Microsoft, Google, and OpenAI. Due to their market position and operational expertise, they are able to command premium pricing and the best margins in the market.

Core Market

Developers

This represents a true developer first and developer friendly environment, where customer onboarding should be almost one click, with minimal friction. Most Neo Cloud providers in this category offer developer friendly consoles, notebooks, and prebuilt applications.

Mid-Market

Mid-Market is a very broad term used for mixed use cases, where the ideal customer profile is not clearly defined and cloud providers are primarily trying to meet demand as it appears. Mid-market not only includes the end customer, but often Neo-Clouds trade GPUs with one another to manage their GPU utilization.

Enterprise

Its well know fact by now that most enterprises, money making business, won’t go with any of Neo-Cloud with their services. Essentially they will be using AI services provided by their Hyperscaler partner, unlike startups, they are not price sensitive. Right now most neo clouds are focused on getting as many as startup customers into their books to keep their fleet utilization high.

Not Neo Cloud

AWS, GCP, Azure, Oracle, xAI

You may notice that I have not included hyperscalers in this discussion. That is intentional. Hyperscalers are not Neo Clouds. They operate in a different market altogether and function with near monopolistic pricing power across developers, enterprises, and government customers.

MyAICompany.ai

This category can be especially confusing, as many AI companies present themselves as offering everything under the sun on their landing pages. For this blog, I have excluded companies that primarily provide API endpoints running on someone else’s hardware, such as Together AI and other Bedrock style offerings. This segment is a large market on its own and is mostly operating expense heavy rather than capital expense heavy.

In case I have misrepresented or missed anything feel free to comment.

Neo-Cloud Product Offering: Picks and Shovels

The core business model for Neo-Clouds remains simple.

They provide the picks and shovels for the AI gold rush. No matter who wins, if anyone does, the demand for their core products is expected to remain strong.

At the same time, many Neo-Clouds aim to grow fast enough to become acquisition targets for larger players or hyperscalers.

Core Product Offerings

• Bare Metal - Bare metal is by far the most popular product offering for many AI startup customers. Most of these startups are either training their own custom models or using off the shelf models from Hugging Face and offering AWS Bedrock style LLM API endpoints. These customers tend to be well funded and usually have access to a healthy end customer pipeline.

• Virtual Machines - Similar to bare metal, but typically offered at a much smaller scale.

• Jupyter Notebooks - Primarily targeted at individual developers and small startups who are experimenting with GPU based AI models.

LLM benchmarking can be confusing, as people often mix up LLM performance testing with benchmarking. Performance testing validates the overall capacity of your server infrastructure, including network latency, CPU performance, and other system-level throughputs. Benchmarking tools, on the other hand, primarily focus on LLM inference engine–specific parameters, which are critical if you are planning to run your own inference platform — something most enterprises are now focusing on.

This is a series of blogs that I will be writing as I go through the process of learning and experimenting with vLLM-based inference solutions, along with insights from real-world use cases operating LLM inference platforms in enterprise environments.

Here are some of the most common inference use cases.

In this example we will be setting up a single node Inference + benchmarking node for experimentation purpose, however, production use case would require the Benchmarking tool should run from a separate node.

For decent benchmarking, you need the following to get started:

• NVIDIA GPU–powered compute platform. This can be your desktop, or you can use any of the Neo Cloud providers. My obvious preference is Denvr Cloud. Feel free to sign up — https://www.denvr.com/

• Hugging Face login. Sign up for a free Hugging Face account. You’ll need it to download models and access gated models such as Meta Llama and others.

• LLM-labs repo. https://github.com/kchandan/llm-labs

Step-by-step guide

To install the necessary packages on the Linux VM (e.g., NVIDIA drivers, Docker, etc.), the easiest approach is to update the IP address in the Ansible inventory file and then let the playbook handle the full installation.

cat llmops/ansible/inventory/hosts.ini
; [vllm_server]
; server_name ansible_user=ubuntu
[llm_workers]
<IP Address> ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/<your_key_file>

Once IP address is update, fire the Ansible playbook to install required packages

(venv) ➜  llmops git:(main) ✗ ansible-playbook -i ansible/inventory/hosts.ini ansible/setup_worker.yml

PLAY [Setup worker nodes] **********************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************
[WARNING]: Host is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
ok: [worker-node]

TASK [docker_install : Update apt and install prerequisites] ***********************************************************************************************************
ok: [worker-node]

TASK [docker_install : Create directory for Docker keyrings] ***********************************************************************************************************
ok: [worker-node]

TASK [docker_install : Download Docker GPG key] ************************************************************************************************************************
ok: [worker-node]

TASK [docker_install : Add Docker repository to apt sources] ***********************************************************************************************************
changed: [worker-node]

TASK [docker_install : Update apt cache after adding Docker repo] ******************************************************************************************************
changed: [worker-node]

TASK [docker_install : Install Docker packages] ************************************************************************************************************************
ok: [worker-node]

TASK [docker_install : Ensure Docker service is enabled and started] ***************************************************************************************************
ok: [worker-node]

TASK [docker_install : Add ubuntu user to docker group] ****************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Download cuda-keyring deb] **********************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Install cuda-keyring deb (dpkg)] ****************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : apt update] *************************************************************************************************************************************
changed: [worker-node]

TASK [nvidia-toolkit : Install cuda-drivers] ***************************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Install prerequisites] **************************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Create keyring directory if missing] ************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Download NVIDIA container toolkit GPG key] ******************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Convert GPG key to dearmor format] **************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Add NVIDIA container toolkit apt repository] ****************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Enable experimental repository (optional)] ******************************************************************************************************
skipping: [worker-node] => (item=deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/experimental/deb/ /)
skipping: [worker-node]

TASK [nvidia-toolkit : Update apt cache after repo add] ****************************************************************************************************************
changed: [worker-node]

TASK [nvidia-toolkit : Install NVIDIA Container Toolkit packages] ******************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Configure NVIDIA Docker runtime] ****************************************************************************************************************
ok: [worker-node]

TASK [nvidia-toolkit : Restart Docker] *********************************************************************************************************************************
changed: [worker-node]

PLAY RECAP *************************************************************************************************************************************************************
worker-node            : ok=22   changed=5    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

Post installation ensure, Driver installation looks good

ubuntu@llmops:~/llm-labs$ nvidia-smi
Sun Jan 11 21:53:01 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01              Driver Version: 590.48.01      CUDA Version: 13.1     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-40GB          Off |   00000000:0A:00.0 Off |                    0 |
| N/A   47C    P0             50W /  400W |       0MiB /  40960MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Create the common docker bridge network so that all containers could talk to each other ( default bridge driver)

docker network create llmops-net

Export the Huggingface token

export HF_TOKEN=hf_token

Now, simply launch the vLLM docker compose, it will take some time to load

ubuntu@llmops:~/llm-labs/llmops/vllm$ docker compose -f docker-compose-vllm-qwen3-0.6B.yml up -d[+] up 1/1 ✔ Container vllm Created                                                                                                                                                                                          0.3subuntu@llmops:~/llm-labs/llmops/vllm$ docker compose -f docker-compose.monitoring.yml up -dWARN[0000] Found orphan containers ([vllm]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.ubuntu@llmops:~/llm-labs/llmops/vllm$ ✔ Container prometheus    Created                                                                                                                                                                                 0.5s ✔ Container dcgm-exporter Created                                                                                                                                                                                 0.5s ✔ Container node-exporter Created                                                                                                                                                                                 0.5s ✔ Container cadvisor      Created                                                                                                                                                                                 0.5s ✔ Container grafana       Created

Ignore the orphan container warning. I have kept those 2 compose file separate deliverable so that more model specific compose files could be added later into the same repo.

Once all containers are downloaded and loaded, it should look like this ( without container crash loop)

ubuntu@llmops:~/llm-labs/llmops/vllm$ docker psCONTAINER ID   IMAGE                             COMMAND                  CREATED              STATUS                    PORTS                                         NAMES750f8e14201d   grafana/grafana:latest            “/run.sh”                58 seconds ago       Up 58 seconds             0.0.0.0:3000->3000/tcp, [::]:3000->3000/tcp   grafana270c865726e9   prom/prometheus:latest            “/bin/prometheus --c…”   59 seconds ago       Up 58 seconds             0.0.0.0:9090->9090/tcp, [::]:9090->9090/tcp   prometheusf679c2313fd2   gcr.io/cadvisor/cadvisor:latest   “/usr/bin/cadvisor -…”   59 seconds ago       Up 58 seconds (healthy)   0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp   cadvisor28873c028c0b   prom/node-exporter:latest         “/bin/node_exporter …”   59 seconds ago       Up 58 seconds             0.0.0.0:9100->9100/tcp, [::]:9100->9100/tcp   node-exporter5e3f54b8f485   nvidia/dcgm-exporter:latest       “/usr/local/dcgm/dcg…”   59 seconds ago       Up 58 seconds             0.0.0.0:9400->9400/tcp, [::]:9400->9400/tcp   dcgm-exporter3b002c0b1d47   vllm/vllm-openai:latest           “vllm serve --model …”   About a minute ago   Up About a minute         0.0.0.0:8000->8000/tcp, [::]:8000->8000/tcp   vllm

Now we have setup the vLLM inference base setup, next step is to setup Nvidia GenAI-Perf

pip install genai-perf

Do a quick test run to see if everything is working

genai-perf profile \  -m Qwen/Qwen3-0.6B \  --endpoint-type chat \  --synthetic-input-tokens-mean 200 \  --synthetic-input-tokens-stddev 0 \  --output-tokens-mean 100 \  --output-tokens-stddev 0 \  --streaming \  --request-count 50 \  --warmup-request-count 10[2026-01-11 23:53:27] DEBUG    Inferred tokenizer from model name: Qwen/Qwen3-0.6B                                                          config_tokenizer.py:79[2026-01-11 23:53:27] INFO     Profiling these models: Qwen/Qwen3-0.6B                                                                         create_config.py:58[2026-01-11 23:53:27] INFO     Model name ‘Qwen/Qwen3-0.6B’ cannot be used to create artifact directory. Instead, ‘Qwen_Qwen3-0.6B’    perf_analyzer_config.py:157                               will be used.[2026-01-11 23:53:27] INFO     Creating tokenizer for: Qwen/Qwen3-0.6B                                                                           subcommand.py:190[2026-01-11 23:53:29] INFO     Running Perf Analyzer : ‘perf_analyzer -m Qwen/Qwen3-0.6B --async --warmup-request-count 10 --stability-percentage subcommand.py:98                               999 --request-count 50 -i http --concurrency-range 1 --service-kind openai --endpoint v1/chat/completions                               --input-data artifacts/Qwen_Qwen3-0.6B-openai-chat-concurrency1/inputs.json --profile-export-file                               artifacts/Qwen_Qwen3-0.6B-openai-chat-concurrency1/profile_export.json’[2026-01-11 23:53:52] INFO     Loading response data from ‘artifacts/Qwen_Qwen3-0.6B-openai-chat-concurrency1/profile_export.json’       profile_data_parser.py:66[2026-01-11 23:53:52] INFO     Parsing total 50 requests.                                                                           llm_profile_data_parser.py:124Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 260.92requests/s]                               NVIDIA GenAI-Perf | LLM Metrics┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┓┃                            Statistic ┃    avg ┃    min ┃    max ┃    p99 ┃    p90 ┃    p75 ┃┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━┩│             Time To First Token (ms) │  12.79 │  11.14 │  16.74 │  15.22 │  13.30 │  13.05 ││            Time To Second Token (ms) │   3.18 │   3.06 │   3.73 │   3.57 │   3.27 │   3.24 ││                 Request Latency (ms) │ 336.79 │ 324.87 │ 348.00 │ 347.84 │ 346.32 │ 345.02 ││             Inter Token Latency (ms) │   3.27 │   3.17 │   3.39 │   3.39 │   3.37 │   3.36 ││     Output Token Throughput Per User │ 305.64 │ 295.21 │ 315.82 │ 315.69 │ 312.30 │ 311.15 ││                    (tokens/sec/user) │        │        │        │        │        │        ││      Output Sequence Length (tokens) │  99.98 │  99.00 │ 100.00 │ 100.00 │ 100.00 │ 100.00 ││       Input Sequence Length (tokens) │ 200.00 │ 200.00 │ 200.00 │ 200.00 │ 200.00 │ 200.00 ││ Output Token Throughput (tokens/sec) │ 296.71 │    N/A │    N/A │    N/A │    N/A │    N/A ││         Request Throughput (per sec) │   2.97 │    N/A │    N/A │    N/A │    N/A │    N/A ││                Request Count (count) │  50.00 │    N/A │    N/A │    N/A │    N/A │    N/A │└──────────────────────────────────────┴────────┴────────┴────────┴────────┴────────┴────────┘[2026-01-11 23:53:52] INFO     Generating artifacts/Qwen_Qwen3-0.6B-openai-chat-concurrency1/profile_export_genai_perf.json                    json_exporter.py:64[2026-01-11 23:53:52] INFO     Generating artifacts/Qwen_Qwen3-0.6B-openai-chat-concurrency1/profile_export_genai_perf.csv

If you are able to see these metrics from GenAI-Perf, it means your setup is complete.

Now let’s move on to setting up the Grafana dashboard.

First, ensure that you have configured the Prometheus backend in Grafana. By default, it points to localhost, so we need to switch it to prometheus, matching the service name used in the Docker Compose file.

As part of the Docker Compose setup, Grafana should automatically pick up the dashboard (NVIDIA + vLLM).

You should now be able to see the metrics flowing into the Grafana dashboard.

At this point, what we have achieved is a basic “hello-world” setup for our LLM benchmarking infrastructure. The next big challenge is to benchmark properly and identify how we can tweak vLLM parameters and GenAI-Perf settings to squeeze the maximum out of the hardware. In this example, I am using a single A100-40GB GPU. It may not sound like much, but these are very powerful cards and work extremely well for agentic workflows where small language models are heavily used.

The next blog will focus more on capturing additional metrics and logs, and on how to get the best out of your hardware.If you are looking to collaborate, please checkout — https://www.becloudready.com/dev-rel-services

References

[1] https://developer.nvidia.com/blog/llm-performance-benchmarking-measuring-nvidia-nim-performance-with-genai-perf/

[2] https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/perf_analyzer/genai-perf/README.html

Demo Night – Build AI Healthcare Apps
Date: December 9, 2025
Location: ShyftLabs, Toronto
Community: TorontoAI

Recordings

Full Recordings

• Demo 1 (FearSense):

• Demo 2 (Moorcheh.ai):

Last week, the TorontoAI community came together for another hands-on Demo Night—this time focused squarely on practical, real-world AI systems, not hype.

Despite snowy weather and a smaller in-person turnout, the evening delivered deep technical insights, candid startup learnings, and live demos that showed what it really takes to move AI from experimentation to production.

The event featured two core demos:

• FearSense — a live walkthrough of a fear-mongering detection application built using Falcons AI models

• Moorcheh.ai — a deep dive into building scalable, production-grade AI assistants and agents using Retrieval-Augmented Generation (RAG)

Why TorontoAI Exists

TorontoAI was created to bridge the gap between developers, startups, and applied AI use cases.

With over 6,000 members across platforms, the community focuses on demo nights, panels, and applied learning—especially for people building and deploying AI systems, not just talking about them.

A special thank-you to ShyftLabs for hosting the event and supporting the local AI ecosystem by opening their space to the community.

Demo #1: FearSense — Detecting Fear-Mongering with Falcons AI

The Problem Being Explored

Fear-driven content is everywhere—news, political speech, social media, and even children’s content. While sentiment analysis is common, fear detection is a more nuanced and under-explored problem, particularly when considering its psychological and societal impact.

This demo explored a simple but important question:

Can we quantify fear in media content, and can that signal be used responsibly—especially in healthcare and research contexts?

About Falcons AI

Falcons AI is a lean, developer-focused AI company whose models consistently rank among the most downloaded on Hugging Face, despite being built by a small, bootstrapped team.

Their success comes from solving specific, real-world problems, rather than chasing generic AGI narratives.

Hugging Face: https://huggingface.co/Falconsai

Model & Architecture Highlights

The FearSense demo leveraged a DistilBERT-based model, fine-tuned specifically for fear-mongering detection.

Unlike large LLMs, this approach:

• Runs efficiently on CPUs

• Requires no GPU acceleration

• Can be deployed locally or on small cloud instances

• Prioritizes determinism and explainability

This makes it particularly suitable for regulated or research-oriented environments.

Live Demo Highlights

The FearSense application was built as a Streamlit app:

GitHub: https://github.com/torontoai-hub/fear-monger-detector

Key capabilities demonstrated:

• Accepts YouTube URLs or raw text transcripts

• Extracts and processes content automatically

• Breaks text into chunks

• Scores each chunk for fear intensity

• Visualizes fear peaks and distributions

Despite real-world demo friction—cloud tunnels, missing Python dependencies, and live debugging—the audience got an honest look at what real AI development looks like: not polished slides, but systems being deployed, debugged, and improved in real time.

Healthcare & Research Implications

One of the most compelling discussions focused on healthcare and research use cases:

• Correlating fear-heavy media consumption with heart rate or stress data from wearables

• Studying impacts on vulnerable populations, including seniors and children

• Providing researchers with tools—not conclusions—to responsibly explore correlation vs. causation

Key takeaway: this was not a medical diagnosis tool, but a research-enabling prototype designed to spark deeper investigation.

Demo #2: Moorcheh.ai — Building Scalable AI Assistants and Agents

The second half of the evening shifted from model-level demos to production AI systems.

The Problem Moorcheh.ai Solves

Many teams struggle to move from:

• A prototype chatbot

to:

• A scalable, accurate, and cost-efficient AI assistant

Common challenges include:

• Complex RAG pipelines

• High latency

• Hallucinations

• Expensive vector databases and re-rankers

• Difficulty exporting prototypes into real applications

Moorcheh.ai positions itself as an infrastructure abstraction layer for AI assistants—reducing complexity while preserving performance and accuracy.

Case Study: DoctorPal AI (Healthcare)

One featured use case was DoctorPal AI, a healthcare assistant built on thousands of pages of medical and nutrition documents.

Using Moorcheh.ai, the team was able to:

• Upload large document sets (PDFs, websites, structured data)

• Automatically chunk, embed, summarize, and index content

• Enforce strict relevance thresholds to prevent hallucinations

• Provide citation-backed responses

• Control which questions the AI is allowed to answer (kiosk mode)

The result was a domain-specific AI assistant that answers questions strictly based on verified source material—not general internet knowledge.

Key Technical Capabilities Demonstrated

• Namespace-based knowledge isolation

• Built-in re-ranking and relevance scoring

• Toggleable kiosk mode to block irrelevant queries

• Model flexibility (Claude, LLaMA, Bedrock-native models)

• API-first design for embedding into real products

• Serverless, cloud-native architecture for cost efficiency

A major differentiator highlighted was the ability to export AI assistants into production applications, rather than remaining locked inside notebooks or playgrounds.

From Chatbots to AI Agents

The demo concluded with an advanced walkthrough of building dynamic AI agents, not just static Q&A bots.

Example shown:

• A legal intake AI agent

• Dynamically adapts questions based on user responses

• Uses RAG for decision rules and instructions—not just documents

• Produces structured summaries for human review

The full workflow—from knowledge base to UI—was assembled in hours, not weeks.

Resources & Integrations

• GitHub (Official SDKs): https://github.com/moorcheh-ai/moorcheh-python-sdk

• LangChain Integration: https://docs.langchain.com/oss/python/integrations/vectorstores/moorcheh

• LlamaIndex Integration: https://developers.llamaindex.ai/python/examples/vector_stores/moorchehdemo/

• MCP Server: https://github.com/moorcheh-ai/moorcheh-mcp

• n8n Integration (Verified Node): https://n8n.io/integrations/moorcheh/

Key Takeaways from the Night

• Small, focused models still matter—especially when accuracy, cost, and deployability are critical

• AI demos should reflect reality: debugging, trade-offs, and iteration

• RAG is no longer optional for serious AI products—but must be implemented carefully

• The future isn’t just chatbots—it’s context-aware, task-driven AI agents

• Community-driven learning accelerates real innovation far more than polished marketing

Thank You & What’s Next

A big thank-you to:

• Falcons AI for sharing their models and philosophy

• Moorcheh.ai for a transparent, in-depth technical walkthrough

• ShyftLabs for hosting

• Everyone who attended and contributed to the discussion

TorontoAI will continue hosting demo nights, panels, and hands-on sessions focused on applied AI, platform engineering, and real-world deployments.

If you’re building something interesting in AI, TorontoAI is your platform.

Stay tuned for upcoming events.

If you enjoy AI Infra Weekly, it would mean the world to me if you invited friends to subscribe and read with us. If you refer friends, you will receive benefits that give you special access to AI Infra Weekly.

How to participate

1. Share AI Infra Weekly. When you use the referral link below, or the “Share” button on any post, you'll get credit for any new subscribers. Simply send the link in a text, email, or share it on social media with friends.

Refer a friend

2. Earn benefits. When more friends use your referral link to subscribe, you’ll receive special benefits.

• Get Free Training material for 3 referrals

• Get Access to Private Slack for 5 referrals

• Get 1-1 30 mins zoom call for 25 referrals

Visit the leaderboard

To learn more, check out Substack’s FAQ.

Thank you for helping get the word out about AI Infra Weekly!

Part I — The Uber Era: A Data-Driven Case Study

3. The Taxi Industry Baseline (2000–2010)

4. The Shock (2015–2017): Taxi Industry Collapse

5. 2018: When Uber Drivers Entered BLS Data

6. Wage Analysis: Nominal vs Real Earnings (2010–2024)

7. Platform Absorption vs Organic Recovery: What Really Happened

8. Data Sources and Methodology

Part II — From Taxi to Talent: AI’s Impact on Knowledge Workers

9. Why Uber Is the Right Lens for Understanding AI

10. Key Differences: Taxi Labor vs Knowledge Work

11. Evidence of AI Productivity Gains (Post-2022 Data)

12. Who Benefits: Platforms, Power Users, or Everyone?

13. The Stenographer Analogy: Skill Compression and Tool Disruption

14. Sector Impact Analysis: Developers, Content, Law, Consulting

15. What the AI Productivity Boom Might Look Like (2025–2030)

Part III — Navigating the Shift

16. Career Strategy in the AI Era (For Individuals)

17. Business Strategy in the AI Era (For Teams & Leaders)

18. What Not to Do: Common Failure Modes

19. Final Thoughts

Part I — The Uber Era: What Really Happened to the Taxi Industry

The Real Uber Story Isn’t About Technology. It’s About Labor.

Uber wasn’t just a transportation company. It was a labor market event. A platform fueled by software, venture capital, and aggressive market incentives collided with a fragmented industry—and reorganized it from the ground up. Most people think Uber “disrupted taxis.” That’s imprecise. Uber consolidated taxi labor, absorbed the workforce, rewrote the compensation model, and shifted economic value away from workers and toward the platform layer.

Before we talk about AI and knowledge workers, we need to understand this story clearly—with data. Because whether you’re a developer, consultant, designer, marketer, or lawyer—everything that happened to taxi drivers is a preview of what unmanaged AI adoption can do to your career path.

The Taxi Industry Baseline (2000–2010)

Before Uber, taxis were a regulated, localized labor system. In most cities, drivers worked under a medallion or license system. The market was artificially supply-constrained—stabilizing wages but preventing competition.

The taxi workforce barely changed from 2000 to 2010—this was a stable labor market. Median pay in 2010 was $11.80/hour ($24,540 annual), according to BLS OEWS.

There was no major change in job count or wage structure. The system was inefficient—but stable.

The Shock (2015–2017): Uber Collapses Taxi Employment

Uber was founded in 2009 and went mainstream between 2013–2016. In that period, taxi demand cratered.

Source: BLS OEWS 2015 & 2017

This was not a gradual evolution—it was a collapse.

Driver Employment Collapse

Taxi jobs dropped 35% in 24 months. Small taxi operators couldn’t compete. Long hours with fixed lease costs made it impossible to earn a living. Drivers began defaulting on loans tied to expensive taxi medallions, especially in New York City.

• NYC taxi medallion prices fell from $1 million to under $150,000 between 2014–2018.

• Hundreds of medallion owners filed for bankruptcy.

• At least 8 NYC taxi drivers died by suicide during the crisis. (NY Times, 2018)

This was not just “tech disruption.” It was a labor displacement crisis.

2018: When Uber Drivers Entered the Dataset

In 2018, something strange happened. Even though taxi jobs had been in free fall, reported employment went up.

The Bureau of Labor Statistics quietly reclassified the occupation to include Uber and Lyft drivers. What looked like an industry “recovery” was really a category expansion.

Taxi Wages: The Hidden Wage Stagnation Story

Let’s look at wages.

Year Median Annual Wage Real Change 2010 $24,540 Baseline 2017 $24,880 Flat 2024 $36,220 Appears higher—but inflation adjusted: flat

Inflation between 2010 and 2024 makes $24,540 equivalent to $35,300 in today’s dollars. So even though nominal wages increased, real wages stayed flat.

What Really Happened: Platform Absorption

Uber didn’t make taxi work better. It changed who captured the value. Instead of small companies owning taxi licenses, Uber aggregated labor under one software marketplace.

Uber rearchitected the labor relationship. Workers lost control, companies collapsed, and earnings stagnated even though productivity (rides per driver) increased.

Welcome to the AI labor platform era.

Part II — From Taxi Drivers to Knowledge Workers: AI Is Replaying the Uber Playbook

Check the chart of Taxi driver salary Vs Uber Stock price ( share holder value )

Uber wasn’t just a transportation story. It was a power shift story—away from labor and toward the platform layer. And this is exactly why it matters now:

AI is doing to knowledge workers what Uber did to taxi drivers.

Not overnight. Not dramatically at first. But mechanistically, it’s the same pattern:

• Fragmented labor markets become platform-coordinated markets.

• Power shifts from individual skill owners → platform orchestrators + tool builders.

• Value accrues upward, while wage pressure increases downward.

• Access to opportunity becomes algorithmically mediated.

Let’s break down the parallels—and the critical differences that will shape who wins and who gets squeezed.

Why Uber Is the Right Lens for Understanding AI

Uber was the first labor shock delivered at internet scale. It bypassed traditional licensing systems, rewrote how supply and demand meet, and moved pricing power away from workers.

AI is following a new but similar pattern:

Platforms transform power—not just productivity. And now AI + SaaS are quietly emerging as labor platforms for white-collar work.

Key Differences: Taxi Drivers vs Knowledge Workers

“AI isn’t Uber for code. It’s Uber for cognition.”

Most important distinction:

Taxi drivers didn’t lose jobs because driving disappeared—they lost because a platform sat between them and demand.

AI is now inserting itself between workers and output.

• Instead of:
  Human → Output

• it becomes:
  Human → AI tool → Output

Whoever controls that middle layer wins.

Evidence: AI Productivity Shift Is Real (Post-2022 Data)

AI isn’t speculation anymore — we now have post-ChatGPT controlled studies.

Inline sources:

• GitHub Copilot Enterprise Study (2023)

• “Generative AI at Work” (MIT, 2024)

• BCG “Jagged Frontier of AI” study (2023)

• Thomson Reuters “Future of Professionals Report” (2024)

But Productivity Gains ≠ Wage Gains

The taxi industry proved something critical:

When productivity rises but bargaining power falls, workers don’t benefit.

This is exactly the early warning signal we’re seeing in knowledge work:

• AI is speeding people up

• Companies are not increasing wages

• Work is shifting to AI tools—not humans

• Platform leverage is increasing

Sound familiar?

AI = Cognitive Uberization

AI is not “automation” in the old sense—it doesn’t just replace tasks. It changes who controls work.

This is no longer speculation—the economic gears are already turning.

The Stenographer Analogy: A Warning from History

Before word processors, every company employed stenographers and typists. It was a career. Then the PC revolution happened—writing became a tool skill, not a profession.

Demand collapsed by over 80% between 1980–2000 (BLS Occupational History). Typewriter jobs didn’t “transition”—they evaporated.

AI is now doing the same to entry-level knowledge work:

• Code scaffolding is automated

• Blog drafting is instant

• Customer support is AI-assisted

• Legal research is AI-accelerated

It won’t remove entire professions—but it will compress their foundations.

Which Knowledge Work Is Hit First?

AI doesn’t remove jobs—it removes levels of work inside job ladders.

This is exactly what Uber did: it didn’t remove driving—it removed the taxi company layer and rewired value flows.

Part III — Navigating the AI Consolidation Wave

If Part I proved what happened to taxi workers in the Uber era, and Part II explained why AI is structurally similar for knowledge work, then Part III is about how to survive it—and win.

AI isn’t just a tool upgrade. It’s a power shift. And just like Uber, it will produce winners and losers — not by talent alone but by strategy.

Let’s get practical.

The Early Career Trap: Why Entry-Level Is in Danger First

AI erodes task-based value, and almost all entry-level work is task-heavy.

Consider:

AI won’t replace people. It replaces tasks, and that compresses jobs from the bottom up. Careers used to be ladders; now they risk becoming trampolines — no stable steps, just jumps for those who adapt.

The AI Moat Framework: Who Wins

There are only five defensible moats in the AI era. Everything else gets commoditized.

If you don’t have one of these moats, you drift toward being a commodity task worker—just like taxi drivers became commodity drivers inside Uber.

Why Most People Will Fail This Transition

Here’s the uncomfortable truth: most people won’t adapt, because this shift isn’t just about learning AI tools — it’s about changing how you work.

Failure modes to avoid:

• Thinking “AI won’t affect my role”

• Treating AI like autocomplete

• Learning tools but not systems

• Refusing to ship faster because of “quality perfection” mindset

• Waiting for corporate training

In disruption cycles, passivity is career suicide.

The Career Strategy for the AI Era (Concrete Playbook)

This is what to do if you want to stay relevant and become unreplaceable.

Step 1: Become AI-native, not AI-curious

• Use AI tools daily: Cursor, GitHub Copilot, Claude, Notion AI, Perplexity

• Treat AI as a pair-programmer or thought partner, not a search engine

• Keep a prompt library + outcome library

Step 2: Move up the value chain fast

• Don’t stay a task worker—graduate to process owner

• Replace grunt work with AI chains and reusable templates

• Example: Instead of writing docs → design doc generator pipeline

Step 3: Build a visible edge

• Publish before/after proof of work

• Solve real problems using AI in public

• Build a LinkedIn/GitHub trail of competence

Step 4: Specialize (do not stay generic)

Pick one:

• Vertical specialization: e.g. AI for healthcare, fintech, real estate

• Workflow specialization: e.g. evals, RAG systems, LLM chain QA

• Role specialization: AI Copilot for legal research, support QA, BI automation

AI favors sharp edges over well-rounded resumes.

Business Strategy: How Companies Avoid Getting “Ubered” by AI

Companies will split into two groups: AI operators and AI tourists.

The 5-Step AI Ops Model

1. Identify 5 repeatable tasks per function (support, HR, eng, ops)

2. Build AI playbooks (prompt + chain + checks)

3. Use evals (accuracy tests + hallucination guardrails)

4. Connect to internal data with secure RAG

5. Ship + measure impact (cycle time, QA rate, velocity)

The Biggest Lesson from the Uber Era

Uber didn’t kill taxis. Taxi companies killed themselves by not adapting. They had a monopoly for decades but never built software leverage. They assumed regulation would protect them. They let a platform sit between them and demand.

Today, if you let AI sit between you and your work, you get commoditized.

But if you learn to own the AI layer, you play from strength—not fear.

The AI era won’t reward the smartest people. It will reward the fastest adapters—people and businesses who turn AI into leverage before it turns them into labor commodities.

The Final Lesson from the Taxi Collapse

Taxi drivers didn’t lose their jobs because people stopped needing rides.
They lost because they didn’t control the platform.
They didn’t build software leverage.
They trusted that “things would stay the same.”

AI is now the platform between you and your output.
The only question is: will you control it or be controlled by it?

Final Call to Action

This is the defining career shift of our generation.

• Don’t be passive. Be intentional.

• Don’t just use AI. Dominate with it.

• Don’t wait for permission. Automate ruthlessly and create leverage.

Because the future won’t belong to the people who fear AI or even to people who can use AI.

The future will belong to the people who can build with AI.

This exclusive panel discussion brings together leaders from technology, retail, finance, and media to explore the opportunities and challenges facing Canada’s AI ecosystem.

Agenda

5:30 – 6:00 PM · Registration, Welcome Snacks & Refreshments

6:00 – 6:15 PM · Introduction of Panelists & Opening Remarks

6:15 – 7:00 PM · Panel Discussion: Topics

• Infrastructure & Power: The capital-intensive nature of AI, GPUs, cloud infrastructure, and Canada’s unique opportunity to lead with clean energy and next-generation data centers.

• AI in Retail: How AI could reshape the grocery industry, online delivery, and the future of jobs in Canada’s retail sector.

• Finance & Regulation: How Canada’s largest banks are adopting AI responsibly, and the role regulation will play in ensuring trust, compliance, and ethical innovation.

• Public Perception & Media: How Canadians view AI, the role of media in shaping public trust, and what better communication could mean for adoption.

7:00 - 7:30 PM: Networking

Panelists

Kyle McCrindle

Kyle McCrindle is the co-founder and Chief Technology Officer (CTO) of Denvr Dataworks, a technology company dedicated to reshaping AI cloud infrastructure to foster accelerated innovation. Kyle has previously led technology teams in Capital Markets trading and enterprise data architecture at major Canadian banks. Under his technical leadership, Denvr Dataworks has developed a purpose-built AI cloud platform that makes HPC & AI technologies more accessible, productive, and cost-efficient.

Mohit Rajhans

Mohit Rajhans is one of Canadas leading media and tech subject matter experts for National Radio and TV outlets. Advisory consultant and board member, jury and festival founder - Frmr TV and Digital Producer. He is a frequent guest on CTV and other local news channels as a Tech and AI expert. He is also recognized for bridging the gap between culture, business, and technology, helping organizations and audiences understand the evolving impact of innovation on society.

Gayatri Sikka

Gayatri is the Sr. Director of Data & Engineering at Loblaws where she leads teams of data scientists and ML engineers to deliver AI-driven products, pricing optimization, and generative AI strategies. With prior experience at Ritual, Facebook, and NerdWallet, they have built scalable data infrastructures, customer segmentation models, and analytics solutions that transformed growth and engagement across retail and fintech. Their expertise bridges machine learning, data strategy, and product innovation, giving them a unique lens on how AI is reshaping industries and how Canada can scale in the global AI race.

Why Attend?

• Gain insights from AI leaders across industries on the future of Canada’s AI ecosystem.

• Network with startup founders, executives, AI professionals, and students shaping the next wave of innovation.

• Understand the intersection of infrastructure, data, regulation, and public trust in scaling AI responsibly.

Who Should Join

• Startup founders & entrepreneurs exploring opportunities in AI

• Decision makers & executives navigating AI adoption in their organizations

• AI professionals & data scientists keen on infrastructure, regulation, and use cases

• Computer science & engineering students looking to connect with industry leaders

Canada has the talent, vision, and leadership — but scaling AI will require collective effort across power, policy, and public trust.

Disclaimer: Event Photography & Filming

By attending this event, you acknowledge and agree that you may be photographed, filmed, or otherwise recorded. Your participation constitutes your consent to the use of such photography, video, or recordings by the event organizers for promotional, marketing, and archival purposes. These materials may be published on our website, social media channels, or other platforms.

TorontoAI is where students, entrepreneurs, and professionals meet to explore the future of Technology. Whether you’re just getting started or scaling an AI startup, TorontoAI is your place to connect, learn, and grow with the community.

Our Past Events

​📲 Connect with us:

• Website: https://www.torontoai.io/

• Instagram: https://www.instagram.com/torontoai_

• ​Facebook: https://www.facebook.com/torontoai

• ​LinkedIn: https://www.linkedin.com/company/torontoai/

• ​TikTok: https://www.tiktok.com/@torontoai

​#TorontoAI #AISummit #MachineLearning # #RAG #MultiAgentSystems #AIInnovation #Networking #SovereignAI

Meet FearSense — a research platform that detects fear-mongering patterns in video transcripts and correlates them with biometric stress data (like heart rate, HRV, and EDA) from wearables such as Fitbit, Apple Health, or Google Fit.

This open demo bridges media analytics and biometric AI, offering an interactive way to explore how emotional content in videos might trigger measurable physiological responses.

Why Study Fear-Mongering?

From news broadcasts to viral social clips, fear-based narratives are everywhere. While these emotional cues drive engagement, they also affect viewers’ stress levels and perception of reality.

FearSense enables you to move beyond subjective opinion — it quantifies both fear intensity in the media content and stress correlation from real-world biometric data.

In other words: it lets you measure the emotional cost of the content you consume.

NLP + Wearable Data = Emotional Insight

FearSense operates as a dual-stream analytics engine

FearSense’s architecture combines modern ML, APIs, and visual analytics:

Under the Hood

• Backend: FastAPI + Celery for async NLP processing

• Data Storage: Postgres/TimescaleDB for temporal data

• ML Model: FalconsAI/fear_mongering_detection

• Frontend: Streamlit (prototype) or Next.js (scalable build)

• Visualization: Plotly & Matplotlib for dual-axis analytics

• APIs: Fitbit, YouTube Transcript API, Whisper (for MP4s)

Experimentation Workflow

• Input a Video Transcript — via YouTube URL, SRT/VTT file, or manual text.

• Run NLP Fear Analysis — segment text and detect fear-mongering intensity.

• Fetch Biometric Data — connect Fitbit or upload CSV.

• Align Streams — correlate emotional spikes with physiological stress.

• Visualize & Export — view interactive charts or export CSV for R/Python.

Example Insights

• See which parts of a news clip evoke the most fear.

• Discover time-lagged correlations between fear intensity and heart rate.

• Identify “fear peaks” that might align with audience stress responses.

Quickstart for Developers

Fork the Repo — https://github.com/torontoai-hub/fear-monger-detector

git clone https://github.com/torontoai-hub/fear-monger-detector.git
cd fear-monger-detector
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Add your Fitbit credentials to .env:

FITBIT_CLIENT_ID=your_client_id_here
FITBIT_CLIENT_SECRET=your_client_secret_here
FITBIT_REDIRECT_URI=http://localhost:8080/callback

cd src/frontend/correlation_engine
streamlit run app.py

Running the Demo

Navigate to the app:

cd src/frontend/correlation_engine
streamlit run app.py

The app will open in your browser at

http://localhost:8501

Run Fear Analysis

Option A: YouTube Video

1. Paste a YouTube URL into the input field

2. The transcript will be fetched automatically

3. Preview appears in the “Transcript Preview” expander

Option B: Manual Input

1. Paste transcript text into the text area

2. Text is immediately ready for analysis

Option C: TED Talks Database

1. Open the TED Talks sidebar expander

2. Sort by title, views, date, or duration

3. Navigate pages and select a talk

4. Click to analyze

Configuration Options

Segmentation Settings (Sidebar)

• Segment Mode: Characters | Sentences | Both

• Max Characters: 200–600 (default: 400)

• Max Sentencess: 1–10 (default: 5)

Analysis Parameters

• Fear Threshold: 0.0–1.0 (default: 0.7)

• Scores above threshold = high risk

• Smoothing Window: 1–10 segments (default: 3)

• Reduces noise in timeline

Chart Options

• Type: Line | Bar | Area chart

• Hover Length: 20–500 characters (default: 30)

Analysis Output

Quick Summary

• Total paragraphs analyzed

• Average fear score vs. threshold

• Peak score and minimum score

• High-risk segment count and percentage

Overall Assessment (Color-coded)

• 🔴 High Risk: Average score ≥ threshold

• 🟡 Moderate: 0.5 ≤ Average score < threshold

• 🟢 Low Risk: Average score < 0.5

Visualizations

• Distribution pie chart (Low / Medium / High)

• Interactive timeline chart (Line / Bar / Area)

• Paragraph-level analysis table

Downloads

• CSV export with all scores and timestamps CSV export with all scores and timestamps

Load Fitbit Heart Rate Data

Prerequisites

• Fitbit account with intraday heart rate data enabled

• Completed OAuth2 authentication (first-time setup)

• Heart rate data recorded on the target date

Steps

1. Navigate to the Fitbit Heart Rate Data section

2. Select the date using the date picker

3. Click the Load Data button

4. Wait for the API response (≈5–10 seconds)

5. View the heart rate chart for the full day

Troubleshooting

• “No data available”

• Ensure you wore your Fitbit device that day

• Check that heart rate tracking was enabled

• Verify OAuth2 token hasn’t expired

• “Invalid credentials”

• Re-run OAuth2 flow

• Check .env file for correct client ID/secret

Align Fear & Heart Rate Streams

Setup Playback Window

1. Navigate to Fear vs. Heart Rate Analysis

2. Configure the time window:

3. Start Time: HH:MM when you started watching

4. Duration: Minutes of video playback (1–30)Click Load Fitbit vs Heart Data

What Happens Next Alignment Process

• Fetch Fitbit data for the specified window (1-min resolution)

• Map transcript segments to relative timestamps

• Interpolate fear scores to match heart rate frequency

• Normalize both streams (0 → 1) for comparison

• Merge datasets on aligned timestamps

Output

• Dual-axis chart: Fear score (line) + heart rate (bars)

• Summary metrics:

• Average fear score

• Average heart rate (bpm)

• Number of aligned data points

• Aligned data table: Preview first 20 rows

• CSV download: Full merged dataset

Use Cases

• Identify if fear spikes correlate with heart rate increases

• Detect delayed physiological responses (lag analysis)

• Compare emotional content with objective stress markers

• Export data for custom statistical analysis in R/Python

Quick Tips & Best Practices

Transcript Quality

• Use complete transcripts with proper punctuation

• Verify timestamps are accurate

• Remove non-verbal annotations like [music] or [applause]

Fitbit Integration

• Ensure intraday API access is enabled

• Wear the device continuously during playback

• Sync Fitbit before fetching data

Analysis Parameters

• Start with the default threshold (0.7) and adjust as needed

• Use smoothing for noisy transcripts

• Compare multiple videos to establish baseline patterns

Correlation Interpretation

• Correlation ≠ causation

• Account for individual baseline differences

• Consider confounding factors (e.g., caffeine, stress, environment)

Use Cases

For Researchers

• Media Studies: Quantify fear-based persuasion in journalism.

• Psychology: Study emotion-to-stress causality in controlled settings.

• Public Health: Examine fear-driven health communication impacts.

For Developers

• Prototype affective computing tools.

• Integrate wearable analytics with NLP pipelines.

• Build emotion-aware applications.

For Individuals

• Understand your emotional responses to media.

• Identify high-stress content and manage exposure.

• Explore mindful media consumption patterns.

Future Directions

FearSense is still a research demo, but the roadmap includes:

• Multi-video batch analysis

• Apple Health

• Real-time streaming analysis

• PDF report generation

• Emotion explainability (LIME/SHAP)

• Multi-language support

• CI/CD with GitHub Actions and Sentry logging

GitHub repo: https://github.com/torontoai-hub/torontoai-llm-lab/tree/main/deepseek-3.2-Exp

Lets first see why so much buzz about DSA and why it is step function of engineering marvel that Deepseek team has delivered.

DeepSeek V3.2 (Exp) — Sparse Attention, Memory Efficiency

DSA replaces full attention O(L²) with a two-stage pipeline:

• Lightning Indexer Head — low-precision (FP8) attention that scores relevance for each token.

• Top-k Token Selection — retains a small subset (e.g. k = 64–128).

• Sparse Core Attention — performs dense attention only on selected tokens.

TL;DR (what finally worked)

Model: deepseek-ai/DeepSeek-V3.2-Exp

Runtime: vLLM (OpenAI-compatible)

Parallelism:

• Tried -dp 8 --enable-expert-parallel → hit NCCL/TCPStore “broken pipe” issues

Stable bring-up: -tp 8 (Tensor Parallel across 8 H200s)

Warmup: Long FP8 GEMM warmups + CUDA graph capture on first run (subsequent restarts are much faster due to cache)

Metrics: vLLM /metrics + Prometheus + Grafana (node_exporter + dcgm-exporter recommended)

Client validation: One-file OpenAI-compatible Python script; plus lm-eval for GSM8K

Grafana: Dashboard parameterized with $model_name = deepseek-ai/DeepSeek-V3.2-Exp

Cloud Provider: Shadeform/Datacrunch/Iceland

Total Cost: $54/2 hours

Details for Developers

Minimum Requirement

As per vLLM recipe book for Deepseek, recommended GPUs are B200 or H200.

Also, Python 3.12 with CUDA 13.

GPU Hunting Strategy

For quick and affordable GPU experiments, I usually rely on shadeform.ai or runpod.ai. Luckily, I had some shadeform.ai credits left, so I used them for this run — and the setup was surprisingly smooth.

First I tried to get B200 node, but I had issues in getting either the BM node available or some cases, could not get nvidia driver working.

shadeform@dawvygtc:~$ sudo  apt install cuda-drivers
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
cuda-drivers is already the newest version (580.95.05-0ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 165 not upgraded.
shadeform@dawvygtc:~$ lspci | grep -i nvidia
17:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
3d:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
60:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
70:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
98:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
bb:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
dd:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
ed:00.0 3D controller: NVIDIA Corporation Device 2901 (rev a1)
shadeform@dawvygtc:~$ nvidia-smi
No devices were found
shadeform@dawvygtc:~$

I could have troubleshooted, but didn’t want to pay $35/hour while I struggle with environment issues. Then I ended up killing the node and look for other node.

H200 + Ubuntu 24 + Nvidia Driver 580 — Worked

Because a full H200 node costs at least $25 per hour, I didn’t want to spend time provisioning Ubuntu 22 and upgrading to Python 3.12. Instead, I looked for an H200 image that already included Ubuntu 24 to minimize setup time. I ended up renting a DataCrunch H200 server in Iceland, and on the first try, the Python and CUDA versions aligned with minimal hassle — so I decided to proceed. It still wasn’t entirely smooth, but the setup was much faster overall.

In order to get pytorch working, you need to follow exact version number. So for Nvidia driver 580, you should use CUDA 13.

Exact step by step guide which you can simply copy can be found in the GitHub Read me — https://github.com/torontoai-hub/torontoai-llm-lab/tree/main/deepseek-3.2-Exp

Install uv to manage to Python dependencies, believe me you will thank me later.

# --- Install Python & pip ---
sudo apt install -y python3 python3-pip
pip install --upgrade pip

# --- Install uv package manager (optional, faster) ---
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

# --- Create and activate virtual environment ---
uv venv
source .venv/bin/activate

# --- Install PyTorch nightly build with CUDA 13.0 support ---
uv pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu130

# Ensure below command return “True” in your Python terminal
import torch
torch.cuda.is_available()

Once aforesaid commands are working, start installing vllm installation

# --- Install vLLM and dependencies ---
uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
uv pip install https://wheels.vllm.ai/dsv32/deep_gemm-2.1.0%2B594953a-cp312-cp312-linux_x86_64.whl

# --- Install supporting Python libraries ---
uv pip install openai transformers accelerate numpy --quiet

# --- Verify vLLM environment ---
python -c “import torch, vllm, transformers, numpy; print(’✅ Environment ready’)”

System Validation script

python3 system_validation.py
======================================================================
SYSTEM INFORMATION
======================================================================
OS: Linux 6.8.0-79-generic
Python: 3.12.3
PyTorch: 2.8.0+cu128
CUDA available: True
CUDA version: 12.8
cuDNN version: 91002
Number of GPUs: 8

======================================================================
GPU DETAILS
======================================================================

GPU[0]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[1]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[2]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[3]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[4]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[5]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[6]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

GPU[7]:
  Name: NVIDIA H200
  Compute Capability: 9.0
  Memory: 150.11 GB
  Multi-Processors: 132
  Status: ✅ Hopper architecture - Supported

Total GPU Memory: 1200.88 GB

======================================================================
NVLINK STATUS
======================================================================
✅ NVLink detected - Multi-GPU performance will be optimal

======================================================================
CONFIGURATION RECOMMENDATIONS
======================================================================
✅ Sufficient GPU memory for DeepSeek-V3.2-Exp
   Recommended mode: EP/DP (--dp 8 --enable-expert-parallel)
(shadeform) shadeform@shadecloud:~$

Here is another catch, as per the vLLM official recipes, it recommends using Expert Parallelism + Data Parallelism (EP/DP), I would not recommend it for H200, unless you have extra time to troubleshoot EP/DP issues.

I would recommend using Tensor Parallel Mode (Fallback) for H200 single full node.

vllm serve deepseek-ai/DeepSeek-V3.2-Exp -tp 8

Downloading the model (what to expect)

DeepSeek-V3.2-Exp has a large number of shards (model-00001-of-000163.safetensors …). With 8 parallel downloads; each shard ~4.30 GB (some ~1.86 GB). With ~28–33 MB/s per stream, 8 at once gives ~220–260 MB/s aggregate (sar showed ~239 MB/s).

What the long warm-up logs mean

You’ll see long sequences like:

• DeepGemm(fp8_gemm_nt) warmup (...) 8192/8192

• DeepGemm(m_grouped_fp8_gemm_nt_contiguous) warmup (W=torch.Size([..., ..., ...]))

• Capturing CUDA graphs (mixed prefill-decode, PIECEWISE/FULL

• vLLM / kernels are profiling & compiling FP8 GEMMs for many layer shapes.

• MoE models do grouped GEMMs

• CUDA Graphs are being captured for common prefill/decode paths to minimize runtime launch overhead.

• The first start is the slowest. Compiled graphs and torch.compile artifacts are cached under:

• ~/.cache/vllm/torch_compile_cache/<hash>/rank_*/backbon– subsequent restarts are much faster.

Maximum concurrency for 163,840 tokens per request: 5.04x

That’s vLLM telling you its KV-cache chunking math and how much intra-request parallelism it can achieve at that context length.

Common bring-up errors & fixes

Symptoms: TCPStore sendBytes... Broken pipe, Failed to check the “should dump” flag, API returns HTTP 500, server shuts down.

Usual causes & fixes:

• A worker/rank died (OOM, kernel assert, unexpected shape) → All ranks try to talk to a dead TCPStore → broken pipe spam.

• Mismatched parallelism vs GPU count → keep it simple: -tp 8 on 8 GPUs; only 1 form of parallelism while stabilizing.

• No IB on the host? → export NCCL_IB_DISABLE=1

• Kernel/driver hiccups → verify nvidia-smi is stable; check dmesg.

• Don’t send traffic during warmup/graph capture; wait until you see the final “All ranks ready”/Uvicorn up logs.

Metrics: Prometheus & exporters

You can simply deploy the Monitoring stack from the git repo

docker compose up -d

You should be able to access the Grafana UI on default user/password ( admin/admin)

http://<publicIP>:3000

You need to add Prometheus data source ( default) and then import the Grafana Dashboard JSON customized for Deepseek V.3.2

Now — Show time

If you see unicorn logs, you can start firing Tests and validation.Final Output

Zero-Shot Evaluation

lm-eval --model local-completions --tasks gsm8k   --model_args model=deepseek-ai/DeepSeek-V3.2-Exp,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=100,max_retries=3,tokenized_requests=False

It could take few minutes to load all the tests

NFO 10-08 01:58:52 [__init__.py:224] Automatically detected platform cuda.
2025-10-08:01:58:55 INFO     [__main__:446] Selected Tasks: [’gsm8k’]
2025-10-08:01:58:55 INFO     [evaluator:202] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2025-10-08:01:58:55 INFO     [evaluator:240] Initializing local-completions model, with arguments: {’model’: ‘deepseek-ai/DeepSeek-V3.2-Exp’, ‘base_url’:
        ‘http://127.0.0.1:8000/v1/completions’, ‘num_concurrent’: 100, ‘max_retries’: 3, ‘tokenized_requests’: False}
2025-10-08:01:58:55 INFO     [models.api_models:170] Using max length 2048 - 1
2025-10-08:01:58:55 INFO     [models.api_models:189] Using tokenizer huggingface
README.md: 7.94kB [00:00, 18.2MB/s]
main/train-00000-of-00001.parquet: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.31M/2.31M [00:01<00:00, 1.86MB/s]
main/test-00000-of-00001.parquet: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 419k/419k [00:00<00:00, 1.38MB/s]
Generating train split: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7473/7473 [00:00<00:00, 342925.03 examples/s]
Generating test split: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [00:00<00:00, 212698.46 examples/s]
2025-10-08:01:59:02 INFO     [evaluator:305] gsm8k: Using gen_kwargs: {’until’: [’Question:’, ‘</s>’, ‘<|im_end|>’], ‘do_sample’: False, ‘temperature’: 0.0}
2025-10-08:01:59:02 INFO     [api.task:434] Building contexts for gsm8k on rank 0...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [00:03<00:00, 402.50it/s]
2025-10-08:01:59:05 INFO     [evaluator:574] Running generate_until requests
2025-10-08:01:59:05 INFO     [models.api_models:692] Tokenized requests are disabled. Context + generation length is not checked.
Requesting API: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [04:55<00:00,  4.47it/s]
fatal: not a git repository (or any of the parent directories): .git
2025-10-08:02:04:03 INFO     [loggers.evaluation_tracker:280] Output path not provided, skipping saving results aggregated
local-completions (model=deepseek-ai/DeepSeek-V3.2-Exp,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=100,max_retries=3,tokenized_requests=False), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 1
|

Final result — which matches with the official doc

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9507|±  |0.0060|
|     |       |strict-match    |     5|exact_match|↑  |0.9484|±  |0.0061|

Few-Shot Evaluation (20 examples)

lm-eval --model local-completions --tasks gsm8k   --model_args model=deepseek-ai/DeepSeek-V3.2-Exp,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=100,max_retries=3,tokenized_requests=False --num_fewshot 20

Result looks pretty good

You can observe the Grafana dashboard for Analytics

Share

It’s important not to be misled by the raw dollar amounts involved. China’s lower cost of capital and the recent restrictions on U.S. GPU technologies, such as Nvidia GPUs, have forced its AI ecosystem to become more resourceful and innovative. This adaptability is evident in breakthroughs like the release of the DeepSeek R1, a system that demonstrates how achieving leadership in the AI race doesn’t always require massive compute power. Instead, efficient and purpose-driven architectures can yield competitive results.

Let’s take a closer look at how these dynamics are shaping AI development in China, examining their strategies, innovations, and how they’re carving out a unique path in this rapidly evolving landscape.

China is rapidly developing its artificial intelligence (AI) capabilities, with AI cloud infrastructure playing a crucial role in this growth. In this article, we’ll explore the current state of AI cloud infrastructure in China, recent investments and build outs, and the reasons for differences in AI cloud infrastructure development between China and the US.

Current State of AI Cloud Infrastructure in China

China’s cloud infrastructure services market is robust and expanding. In the third quarter of 2024, spending in this sector reached $10.2 billion, reflecting an 11% year-on-year increase1. This growth is significantly driven by the increasing integration of AI capabilities into cloud-based applications across various industries1. Alibaba Cloud, Huawei Cloud, and Tencent Cloud are the dominant players in this market, collectively holding a 70% market share1.

Benchmarking AI Models

Chinese firms are actively working to improve their AI models by focusing on several key benchmarks:

• Multi-modal understanding and generation: This involves developing AI models that can process and generate content across different modalities, such as text, images, audio, and video.

• Increased context windows: Expanding the context windows of AI models allows them to process and remember more information, leading to more comprehensive and nuanced understanding.

• Cross-modal reasoning: This focuses on enabling AI models to reason and draw connections across different modalities, such as understanding the relationship between a text passage and an associated image.

These efforts demonstrate China’s commitment to advancing the technical capabilities of its AI models.

Despite facing challenges, such as US technology sanctions that restrict access to advanced semiconductor chips, China has demonstrated remarkable resilience and innovation in AI development. To overcome these limitations, Chinese engineers are optimizing AI workflows through techniques like model pruning and knowledge distillation. These techniques reduce the size and complexity of AI models while maintaining accuracy, enabling them to operate efficiently on less-advanced hardware3.

China is also actively developing domestic hardware alternatives to reduce reliance on foreign suppliers. For example, Huawei has made significant strides in designing and producing AI-specific chips, such as the Ascend series3. This strategic approach highlights China’s determination to achieve technological independence in the AI domain.

Furthermore, China is actively promoting AI+ initiatives and integrating AI technology across various sectors, including healthcare, education, finance, and urban management4. By the first quarter of 2024, mainland China hosted over 4,500 AI companies, representing 15% of the global total. Major cities like Beijing, Shanghai, and Shenzhen are at the forefront of AI development, each specializing in different segments of the AI value chain4.

The increasing number of enterprise deployments of AI solutions in China further indicates the growing adoption and integration of AI across various industries. This trend is expected to continue as AI technology matures and demonstrates its potential to enhance productivity and efficiency.

Recent Investments in AI Cloud Infrastructure in China

Chinese tech companies are making substantial investments in AI cloud infrastructure to strengthen their position in the global AI landscape. Here are some notable examples:

• ByteDance is planning to invest over $12 billion globally in AI infrastructure in 2025. This investment will be used to acquire AI chips within China and bolster its AI model training overseas5. This move is motivated by ByteDance’s ambition to strengthen its technological autonomy and competitiveness on the global stage, particularly in light of US export controls and pressure to divest TikTok.

• Alibaba has announced plans to significantly increase short-term investment in AI-related infrastructure to meet the rising demand for cloud computing and API services essential for AI model deployment1.

• Huawei launched the Huawei Cloud AI for Startups program, offering AI startups up to CNY1.2 million (around US$165,000) in cloud computing resources, reinforcing its commitment to ecosystem development and technological leadership.

• Tencent has seen a surge in AI demand that has significantly boosted its GPU-related revenue.

These investments highlight the commitment of Chinese tech giants to maintain their AI lead at home and compete on a global stage1.

Recent Build Outs of AI Cloud Infrastructure in China

China is actively building out its AI cloud infrastructure to support the growing demand for AI applications. In August 2024, the Chinese government announced a 43.5 billion yuan (US$6.1 billion) initiative to build eight new computing hubs7. These hubs will provide the necessary computing power for AI model training and deployment.

Chinese companies are also investing in cloud computing and distributed computing infrastructure to enable AI models to scale using networks of lower-performance processors3. This approach helps to overcome the challenges posed by limited access to advanced chips.

Furthermore, Chinese cloud providers are developing AI model development platforms that allow third-party open-source AI models to be integrated8. This openness fosters collaboration and innovation within the AI ecosystem.

Examples of recent build outs include:

• Alibaba Cloud’s Bailian platform, a one-stop AI foundation model development platform launched in October 2024.

• Huawei’s L4 autonomous driving network solution — Xinghe AI autonomous driving network, launched in September 2024.

However, this rapid expansion of AI cloud infrastructure also presents challenges. Cloud infrastructure resources are becoming increasingly scarce, and cloud vendors are facing rising investment costs. This situation underscores the need for efficient resource allocation and cost optimization strategies in AI cloud infrastructure development.

Reasons for Differences in AI Cloud Infrastructure Investments and Build Outs Between China and the US

While both China and the US are investing heavily in AI cloud infrastructure, there are some key differences in their approaches.

Investment Focus

• The US leads in global AI investment, with a focus on frontier AI research and development10. American companies are investing in cutting-edge technologies and applications, such as autonomous vehicles and predictive analytics11. This leadership is largely attributed to the investment of private capital and innovations by American companies of all sizes.

• China, on the other hand, is focusing on narrower AI applications, particularly machine vision. This focus is partly driven by the country’s need to address specific challenges, such as optimizing energy consumption and improving crop yields.

Hardware Access

• The majority of global AI compute capacity is concentrated in the United States, and U.S. compute providers already dominate the global market for cloud infrastructure14. The US also has a significant advantage in AI computing power due to its access to the most advanced GPUs. This advantage is reflected in the concentration of high-end AI chips in US data centers.

• US export controls have limited China’s access to advanced AI chips, forcing Chinese companies to adopt innovative approaches to optimize AI workflows and develop domestic hardware alternatives3.

Government Support

• The Chinese government is actively promoting AI development through initiatives like the “Made in China 2025” plan and the construction of new computing hubs. This support helps to accelerate the growth of the AI industry in China.

• In the US, government support for AI is also strong, with initiatives like the National AI Research Resource pilot and export controls to maintain US leadership in AI.

Challenges

AI companies in China face two critical stumbling blocks in their pursuit of profitability:

• Regulatory environment: The evolving regulatory landscape for AI in China presents challenges for companies navigating compliance requirements and uncertainties.

• Access to computing power: Limited access to advanced chips due to US export controls poses a significant challenge for Chinese AI companies that require high-performance computing for model training and development.

These challenges highlight the need for continued innovation and adaptation within the Chinese AI industry.

Conclusion

China’s investment in and build out of AI cloud infrastructure is a strategic priority driven by its ambition to become a global leader in AI. Despite challenges, such as limited access to advanced chips, China has shown resilience and innovation in AI development. The country’s focus on narrower AI applications, coupled with strong government support, is helping to accelerate the growth of its AI industry.

While the US currently leads in AI investment and hardware access, China is rapidly catching up. This competition between the two countries is likely to intensify in the coming years, driving further innovation and development in AI cloud infrastructure.

The implications of China’s AI cloud infrastructure development are significant. It has the potential to reshape the global AI landscape, creating new opportunities and challenges for businesses worldwide. As China continues to invest in AI, it is crucial to consider the ethical implications of AI development and ensure responsible use of this technology. The development of domestic hardware alternatives and the focus on optimizing AI workflows demonstrate China’s determination to overcome limitations and achieve technological independence. This drive could lead to a more multipolar global AI landscape, with China playing a more prominent role in shaping the future of AI.

In the past few years, Software-as-a-Service (SaaS) providers have become the backbone of businesses worldwide. From managing customer relationships to streamlining internal processes, SaaS tools are indispensable. However, for many Canadian businesses, a troubling trend has emerged: SaaS inflation. Rising subscription costs, compounded by a weaker Canadian dollar, have left many companies grappling with tighter margins and difficult choices.

This impact is not uniform across all businesses. Those generating income in USD have a natural hedge against this inflation, while companies relying solely on Canadian revenue face a tougher battle.

Understanding SaaS Inflation

The inflation in SaaS pricing stems from multiple factors:

1. Rising Operational Costs: SaaS providers are dealing with their own increasing costs, from data center expansions to employee salaries.

2. Currency Exchange Rates: Most SaaS providers price their offerings in USD, which has seen significant fluctuations against the Canadian dollar.

3. Tiered Pricing Models: Many SaaS companies are shifting to tiered models that emphasize "per-user" pricing, inflating costs as teams grow.

For Canadian businesses reliant on SaaS tools, these factors combine to create a perfect storm of rising expenses.

The Income-Expense Mismatch for Canadian Businesses

Businesses with CAD Income

Companies operating exclusively within Canada face a unique challenge. While their income remains in CAD, SaaS subscriptions are often priced in USD. With the Canadian dollar trading lower against the USD, these businesses see their effective costs rising without a corresponding increase in revenue.

For example:

• A SaaS tool priced at $100 USD/month cost around $130 CAD when the exchange rate was 1.3. Now, with an exchange rate of 1.37, the same tool costs $137 CAD/month. Over a year, that small difference adds up significantly.

Businesses with USD Income

In contrast, companies with a sizable portion of their revenue coming from USD clients are much better positioned. Since they’re earning in the same currency as their SaaS expenses, they are insulated from the adverse effects of exchange rate fluctuations. In fact, these businesses might even benefit from the weaker CAD when converting their USD income back to CAD.

Case Studies: Winners and Losers

Case 1: A Local Marketing Agency

A Calgary-based marketing agency, operating entirely within Canada, heavily relies on SaaS tools like HubSpot, Asana, and Adobe Creative Cloud. Over the last three years, their SaaS costs have risen by 25%, but their client fees—charged in CAD—haven't changed significantly. Margins have shrunk, forcing the agency to consider downgrading their SaaS subscriptions or passing costs onto clients.

Case 2: A Tech Startup with Global Clients

A Toronto-based AI startup earns 70% of its revenue from U.S. clients. Their SaaS costs have risen similarly, but their income in USD means the exchange rate has worked in their favor. They’ve been able to weather SaaS inflation without significant changes to their operational budget.

Strategies for Coping with SaaS Inflation

1. Negotiate with Vendors: Many SaaS providers offer discounts for annual subscriptions or can lock in pricing for multi-year contracts. Don’t hesitate to negotiate, especially if you’re a long-term customer.

2. Explore Alternative Tools: Open-source or Canadian SaaS providers might offer comparable functionality without the penalty of currency conversion.

3. Optimize Usage: Audit your SaaS stack regularly to eliminate underused tools or unused licenses.

4. Diversify Revenue Streams: Businesses can explore international markets to gain USD-based income, providing a natural hedge against currency fluctuations.

The Path Forward

For Canadian businesses, SaaS inflation is an undeniable challenge, but not an insurmountable one. Companies relying solely on CAD income must be proactive in managing costs and seeking efficiencies. On the other hand, those with significant USD income can view this as an opportunity to solidify their market position.

As SaaS continues to dominate the business landscape, ensuring that your revenue streams and cost structures are aligned with global economic realities will be key to staying competitive. In a world where even software can inflate, adaptability is the best tool in your arsenal.

Subscribe now