Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Apr 22, 2013

• modified approach to including elements with class ingress that are outside of the main article

  

  Hallvord Reiar M. Steen

  committed
  22ca149

  Copy full SHA for 22ca149

Commits on Apr 16, 2013

• class name tweaks

  

  Hallvord Reiar M. Steen

  committed
  8db40ec

  Copy full SHA for 8db40ec

• when looking at nearby elements for text nodes that might be worth including, go up to grandparent level rather than just parent

  

  Hallvord Reiar M. Steen

  committed
  cdb0ea5

  Copy full SHA for cdb0ea5

Commits on Mar 14, 2013

• fix indentation bug

  

  Hallvord Reiar M. Steen

  committed
  acdef11

  Copy full SHA for acdef11

Commits on Mar 1, 2013

• make sanitize() remove less content if not in 'ruthless' mode (first pass)

  

  Hallvord Reiar M. Steen

  committed
  5d85869

  Copy full SHA for 5d85869

• find SPAN that contains BR but is not inside P, convert them to P too

  

  Hallvord Reiar M. Steen

  committed
  668612c

  Copy full SHA for 668612c

• further regexp tweaks, remove more 'social' and 'tools' stuff but don't scan for the word 'tools' only

  

  Hallvord Reiar M. Steen

  committed
  4c2a963

  Copy full SHA for 4c2a963

• rank elements marked itemprop=articleBody highly

  

  Hallvord Reiar M. Steen

  committed
  29a4893

  Copy full SHA for 29a4893

• For future compatibility, compare elements to None rather than relying on their false/true-ness

  

  Hallvord Reiar M. Steen

  committed
  c3a59bd

  Copy full SHA for c3a59bd

Commits on Feb 28, 2013

• revert 2d63abe27a - caused bugs

  

  Hallvord Reiar M. Steen

  committed
  e472778

  Copy full SHA for e472778

• further classname tweaking

  

  Hallvord Reiar M. Steen

  committed
  92cdd95

  Copy full SHA for 92cdd95

Commits on Feb 27, 2013

• look for itemprop=mainContentOfArticle when weighting elements

  

  Hallvord Reiar M. Steen

  committed
  9881cec

  Copy full SHA for 9881cec

Commits on Feb 15, 2013

• more tweaking of regexp for class/id filtering

  

  Hallvord Reiar M. Steen

  committed
  ee02ed2

  Copy full SHA for ee02ed2

• more tweaking of regexp

  

  Hallvord Reiar M. Steen

  committed
  be36c64

  Copy full SHA for be36c64

• if neither ruthless nor lenient parsing gives us content, fall back to naively extracting paragraph tags

  

  Hallvord Reiar M. Steen

  committed
  5772d04

  Copy full SHA for 5772d04

• more debug logging of non-ruthless switch

  

  Hallvord Reiar M. Steen

  committed
  1d99efb

  Copy full SHA for 1d99efb

• self.sanitize() could remove less content if not in 'ruthless' mode

  

  Hallvord Reiar M. Steen

  committed
  4cd3f71

  Copy full SHA for 4cd3f71

• self.sanitize() should not modify its argument 'node'. Let's make a copy..

  

  Hallvord Reiar M. Steen

  committed
  2d63abe

  Copy full SHA for 2d63abe

• tweak content regexpes further, better removal of comment form prose and FB junk

  

  Hallvord Reiar M. Steen

  committed
  cec1c4b

  Copy full SHA for cec1c4b

• remove all empty tags except small whitelist (br, img, hr)

  

  Hallvord Reiar M. Steen

  committed
  8b2494b

  Copy full SHA for 8b2494b

• remove A elements with rel=tag and parent element

  

  Hallvord Reiar M. Steen

  committed
  873b4ec

  Copy full SHA for 873b4ec

• remove 'get Flash player' links with ancestors

  

  Hallvord Reiar M. Steen

  committed
  b983180

  Copy full SHA for b983180

Commits on Feb 14, 2013

• drop empty A tags to prevent self-closing serialization (should be all inlines perhaps?)

  

  Hallvord Reiar M. Steen

  committed
  f8c4672

  Copy full SHA for f8c4672

• reverting d719a48bcb8782673f8d826c34d2b22113216c38 - seems to work even without it, and the code didn't do what I intended anyway

  

  Hallvord Reiar M. Steen

  committed
  6c9d8f8

  Copy full SHA for 6c9d8f8

• clean NOSCRIPT and some other tags

  

  Hallvord Reiar M. Steen

  committed
  abeb421

  Copy full SHA for abeb421

• adding italian class name to positiveRe

  

  Hallvord Reiar M. Steen

  committed
  ca59730

  Copy full SHA for ca59730

• lowr score for elements whose id or class names match unlikelyCandidatesRe, fixes extracts from blog posts with long comments getting a high score

  

  Hallvord Reiar M. Steen

  committed
  1c7ae28

  Copy full SHA for 1c7ae28

• using class_or_id_matches method

  

  Hallvord Reiar M. Steen

  committed
  aba4e64

  Copy full SHA for aba4e64

• added class_or_id_matches method

  

  Hallvord Reiar M. Steen

  committed
  075c22f

  Copy full SHA for 075c22f

• tweaking regexp rules for class/id

  

  Hallvord Reiar M. Steen

  committed
  a5a6247

  Copy full SHA for a5a6247

• increase minimum text length to 28 characters, found to work better

  

  Hallvord Reiar M. Steen

  committed
  1939e99

  Copy full SHA for 1939e99

• fall back to using html5parser if lxml.html parser fails

  

  Hallvord Reiar M. Steen

  committed
  d719a48

  Copy full SHA for d719a48

• attempting a more robust encoding handling

  

  Hallvord Reiar M. Steen

  committed
  ef45864

  Copy full SHA for ef45864

• remove META tags too, avoid trouble with META refresh

  

  Hallvord Reiar M. Steen

  committed
  ea444ba

  Copy full SHA for ea444ba

Commits on Oct 17, 2012

• Merge pull request #29 from hush-hush/master

  Show description for 4e3192f
  buriy

  committed
  4e3192f

  Copy full SHA for 4e3192f