Skip to content

yoji0806/paper_scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is a child project of Paper in Economics.

paper scraper

A Python code to do a web scraping of academic papers and data from various sites.

about Directory & Files

  • sample : store sample python code in .ipynb format.
  • *.py : Any python file in the root directory is almost identical to the cloud's running code.

※note:

*.py in the root directory is not 100% the same as the cloud's running code because DB-related codes is hidden for security.
If you will implement another web scraping code, please follow scrapers_journal_aer.py structue.
It should have DOI as a variable, and the return type should be dict.

Potential Bugs

Please refere potentialBugs.md.

Style

Docstrings

Please follow Google Style Python Docstrings.

Future Work (no particular order)

  • ☆ extract references of each paper.
  • collect lectures of Economics.
  • make & open a spreadsheet that everyone can put materials on. -> move this spreadsheet to DB in the cloud.
  • add other journals as resources.
  • collect non-paper information in economics like, open-data, experimental data, about a professor...etc
  • Reccomend feature.
  • Note feature.
  • Export feature.

About

a python code to do a web scraping academic papers and data from various sites.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages