This project is a Node.js script that clones a website by downloading its HTML, CSS, JavaScript, and image resources. It's designed to create a local copy of a website for offline viewing or analysis.
- Downloads and saves the main HTML content of a specified URL
- Retrieves and stores all linked CSS stylesheets
- Captures and saves all referenced JavaScript files
- Downloads all images found on the page
- Maintains the original structure of resources in local directories
- Generates unique filenames to avoid conflicts
Before you begin, ensure you have met the following requirements:
- Node.js installed on your local machine
- npm (Node Package Manager) to install dependencies
-
Clone this repository to your local machine:
git clone https://github.com/yourusername/website-cloner.git -
Navigate to the project directory:
cd website-cloner -
Install the required dependencies:
npm install
-
Open the
main.jsfile and modify the URL in the last line to the website you want to clone:downloadWebpage('https://example.com')
-
Run the script:
node main.js -
The cloned website will be saved in the
websitesdirectory, organized by domain name.
main.js: The main script that handles the website cloning processwebsites/: Directory where cloned websites are stored[domain]/: Subdirectory for each cloned website (e.g.,example.com/)index.html: The main HTML file of the cloned websitecss/: Directory containing downloaded CSS filesjs/: Directory containing downloaded JavaScript filesimg/: Directory containing downloaded image files
- axios: Promise-based HTTP client for making requests
- cheerio: Fast, flexible & lean implementation of core jQuery for parsing HTML
- This script captures the static content of a website. Dynamic content loaded via JavaScript may not be fully captured.
- It does not follow links to other pages within the website.
- Some websites may have measures in place to prevent scraping, which could affect the cloning process.
Contributions to improve the Website Cloner are welcome. Please follow these steps:
- Fork the repository
- Create a new branch (
git checkout -b feature/amazing-feature) - Make your changes
- Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is open source and available under the MIT License.