Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Jul 13, 2015

• default to scrapers in spendright/

  

  David Marin

  committed
  468416e

  Copy full SHA for 468416e

Commits on Jun 1, 2015

• create scraper table if needed

  

  David Marin

  committed
  85e0c37

  Copy full SHA for 85e0c37

• orphaned fix for clarification of claims

  

  David Marin

  committed
  4c055d4

  Copy full SHA for 4c055d4

Commits on May 1, 2015

• there are no longer records for categories

  

  David Marin

  committed
  ef48c91

  Copy full SHA for ef48c91

Commits on Apr 30, 2015

• fix harness to allow empty brand field, clear obsolete tables

  

  David Marin

  committed
  1990b74

  Copy full SHA for 1990b74

Commits on Mar 6, 2015

• removed obsolete tables. NEED TO UPDATE SCRAPERS BEFORE USING THIS VERSION

  

  David Marin

  committed
  91f7b88

  Copy full SHA for 91f7b88

• added merged tables for brand/company

  

  David Marin

  committed
  f0ed4a1

  Copy full SHA for f0ed4a1

Commits on Mar 5, 2015

• add "subcategory" table

  

  David Marin

  committed
  846d4da

  Copy full SHA for 846d4da

Commits on Dec 6, 2014

• common code for trimming off useless subjects from sentences

  

  David Marin

  committed
  f01af29

  Copy full SHA for f01af29

• sentence splitting, improved recognition of poor transparency

  

  David Marin

  committed
  0031995

  Copy full SHA for 0031995

Commits on Dec 5, 2014

• another fix to scraper harness

  

  David Marin

  committed
  e3b09f6

  Copy full SHA for e3b09f6

• factor out all the code for deciding when to run a scraper

  

  David Marin

  committed
  9baabb4

  Copy full SHA for 9baabb4

• whoops, always run scrapers in whitelist

  

  David Marin

  committed
  8ec1763

  Copy full SHA for 8ec1763

• added documentation to run_scrapers(), added packages argument

  

  David Marin

  committed
  53d9c1f

  Copy full SHA for 53d9c1f

• force re-running scrapers from code

  

  David Marin

  committed
  57e9b05

  Copy full SHA for 57e9b05

Commits on Nov 29, 2014

• attempt to handle climate_counts claims

  

  David Marin

  committed
  6fbc9a4

  Copy full SHA for 6fbc9a4

• add code to judge and clarify claims

  

  David Marin

  committed
  091f882

  Copy full SHA for 091f882

Commits on Nov 21, 2014

• force decoding of pages in UTF-8 if possible

  

  David Marin

  committed
  8720884

  Copy full SHA for 8720884

• added support for ISO dates as well as datetimes

  

  David Marin

  committed
  c8d42a7

  Copy full SHA for c8d42a7

• allow POST requests

  

  David Marin

  committed
  84bf44d

  Copy full SHA for 84bf44d

• fix crawl-delay debug printout

  

  David Marin

  committed
  bba3272

  Copy full SHA for bba3272

• Merge branch 'master' of github.com:spendright-scrapers/srs

  

  David Marin

  committed
  0f767c7

  Copy full SHA for 0f767c7

• added campaign_*_claim tables

  

  David Marin

  committed
  1c13125

  Copy full SHA for 1c13125

Commits on Nov 4, 2014

• moved smunch() into srs

  

  David Marin

  committed
  22159d7

  Copy full SHA for 22159d7

Commits on Nov 2, 2014

• Merge branch 'master' of github.com:spendright-scrapers/srs

  

  David Marin

  committed
  31d8fa6

  Copy full SHA for 31d8fa6

• use transactions to *actually* delete old data

  

  David Marin

  committed
  3a000dd

  Copy full SHA for 3a000dd

Commits on Oct 27, 2014

• use get_db_path() rather than DB_FILE_EXT

  

  David Marin

  committed
  7466dff

  Copy full SHA for 7466dff

Commits on Oct 17, 2014

• attempt to stop log messages

  

  David Marin

  committed
  c29bc67

  Copy full SHA for c29bc67

• added dependency for requests

  

  David Marin

  committed
  3385ffc

  Copy full SHA for 3385ffc

• whoops, that's "requests"

  

  David Marin

  committed
  f2da0d9

  Copy full SHA for f2da0d9

• don't show log lines for requests module by default

  

  David Marin

  committed
  ba41c74

  Copy full SHA for ba41c74

• scrape() now parses robots.txt

  

  David Marin

  committed
  720f1f6

  Copy full SHA for 720f1f6

Commits on Oct 16, 2014

• added reppy 0.3.0 to srs/vendor (morph.io doesn't support recursive submodules)

  

  David Marin

  committed
  6f23e34

  Copy full SHA for 6f23e34

Commits on Oct 13, 2014

• Merge branch 'master' of github.com:spendright-scrapers/srs

  

  David Marin

  committed
  469f964

  Copy full SHA for 469f964

• broke out iso datetime code

  

  David Marin

  committed
  6f68153

  Copy full SHA for 6f68153