NEPSE is a personal project that fetches live LTP (last traded price) and corporate actions (bonus, rights share, cash dividend) data for stocks listed on NEPSE (Nepal Stock Exchange).

WARNING / IMPORTANT: Educational-only and ethical use

• This project is provided strictly for educational, research, and personal learning purposes only.
• Respect website Terms of Service and robots.txt. If a data provider offers an official API, prefer that API and obtain permission for automated access.
• The author is not responsible for how others use this code. Use at your own risk.

What this project does

• Provides utilities and API endpoints to fetch:
  • Live LTP for symbols on NEPSE (example scraping of Sharesansar or other public sources).
  • Corporate actions over the last 5 years (bonus issues, rights issues, cash dividends).
• Supports automated refresh (e.g., scheduled refresh every 2 days) and manual refresh endpoints/actions.
• Provides parsing and normalization so downstream tools can consume the data.

High-level architecture

• Next.js / TypeScript API routes for web-accessible endpoints.
• Scraper modules that fetch and parse data:
  • Lightweight HTML parsing (node-fetch + jsdom) for static content.
  • Optionally, headless browser approach (Puppeteer/Playwright) for pages that render dynamically or rely on JavaScript.
• Optional Python helpers and scripts for heavier data processing or archival tasks.
• Cache layer / DB (recommended) for storing last-known values and corporate actions to avoid refetching every request.
• Scheduler (cron, GitHub Actions, or an external job runner) to refresh corporate actions every 2 days.

Design & implementation notes

• Caching: Store results in a local DB (SQLite/Postgres) or an in-memory cache for live LTP to avoid repeated heavy fetches.
• Respect robots.txt: Check each data source's robots.txt and terms of service before scraping.
• Avoid bypass instructions: This repo intentionally does NOT include instructions to bypass anti-scraping protections (CAPTCHA, IP bans, Cloudflare IUAM, or other measures). Attempting to evade such protections may be illegal or violate terms of service.

Data refresh strategy

• Corporate actions (bonus/rights/cash dividend):
  • Default: refresh every 48 hours (cron or scheduled job).
  • Manual: expose an authenticated endpoint to trigger refresh.
• Live LTP:
  • Ideally use a streaming or official endpoint. If unavailable, avoid aggressive polling.
  • Aggregate and store snapshots; provide a “latest known” endpoint.

Credits & sources

• Data sources that may be referenced:
  • NepaliPaisa (https://nepalipaisa.com/)
  • MeroLagani (https://merolagani.com/)
  • ShareSansar (https://www.sharesansar.com/)

Contributing

• Contributions are welcome, but please:
  • Do not add code that circumvents a site's security or denies service.
  • Open PRs that improve parsing resilience, add official API support, or improve caching/rate-limiting.
  • Include tests for parsers and keep changes well-documented.

License

• This repository does not grant permission to scrape any website. It provides code examples and utilities for educational purposes only.