rs.io

Books on probabilistic programming

Thu, 10 Oct 2019 10:43:00 -0500

Model-Based Machine Learning

For more than ten years we have been working on such a software framework at Microsoft Research, called Infer.NET [Minka et al., 2014].

Bayesian Data Analysis using Probabilistic Programs–Statistics as pottery

Learning statistics is like learning pottery. With pottery, you can learn how to make different shapes (e.g. a bowl, a vase, a spoon) without understanding general principles. The other way is to learn the basic strokes of forming pottery (e.g. how to mold a curved surface, a flat surface, long pointy things). In this course, we are going to learn the basic strokes of statistics, and compose those strokes to make shapes you’ve seen before (e.g. a t-test), some shapes you’ve probably never seen before, and develop ideas how you would make new shapes if you needed to. We won’t learn what tests apply to what data types but instead foster the ability to reason through data analysis. We will do this through the lens of Bayesian statistics, though the basic ideas will aid your understanding of classical (frequentist) statistics as well.

Probabilistic Models of Cognition

This book explores the probabilistic approach to cognitive science, which models learning and reasoning as inference in complex probabilistic models. We examine how a broad range of empirical phenomena, including intuitive physics, concept learning, causal reasoning, social cognition, and language understanding, can be modeled using probabilistic programs (using the WebPPL language).

The Design and Implementation of Probabilistic Programming Languages

This book explains how to implement PPLs by lightweight embedding into a host language. We illustrate this by designing and implementing WebPPL, a small PPL embedded in Javascript. We show how to implement several algorithms for universal probabilistic inference, including priority-based enumeration with caching, particle filtering, and Markov chain Monte Carlo. We use program transformations to expose the information required by these algorithms, including continuations and stack addresses. We illustrate these ideas with examples drawn from semantic parsing, natural language pragmatics, and procedural graphics.

An Introduction to Probabilistic Programming

This document is designed to be a first-year graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understanding of either or, ideally, both probabilistic machine learning and programming languages.

Bayesian Methods for Hackers

Bayesian Methods for Hackers is designed as a introduction to Bayesian inference from a computational/understanding-first, and mathematics-second, point of view. Of course as an introductory book, we can only leave it at that: an introductory book. For the mathematically trained, they may cure the curiosity this text generates with other texts designed with mathematical analysis in mind. For the enthusiast with less mathematical-background, or one who is not interested in the mathematics but simply the practice of Bayesian methods, this text should be sufficient and entertaining.

Practical Probabilistic Programming

Practical Probabilistic Programming introduces the working programmer to probabilistic programming. In it, you’ll learn how to use the PP paradigm to model application domains and then express those probabilistic models in code. Although PP can seem abstract, in this book you’ll immediately work on practical examples, like using the Figaro language to build a spam filter and applying Bayesian and Markov networks, to diagnose computer system data problems and recover digital images.

Stan: A Probabilistic Programming Language

Probabilistic programming in Python using PyMC3

Bonus video: A Personal Viewpoint on Probabilistic Programming

Fixing "FATAL:udev_loader.cc(38) Check failed: false." errors on NixOS

Wed, 09 Oct 2019 16:31:00 -0500

This is a problem that I ran into while figuring out how to get Pixelator running–the issue is that electron apps expect to be linked against the old libudev.so.0, not the new libudev.so.1.

The error itself looks like

Start init.
[1654:1009/144147.908343:FATAL:udev_loader.cc(38)] Check failed: false.
#0 0x00000378afa7 <unknown>
#1 0x0000037cb27b <unknown>
#2 0x000001109cd3 <unknown>
#3 0x000001109626 <unknown>
#4 0x000001109d50 <unknown>
#5 0x000003b68084 <unknown>
#6 0x0000037a2370 <unknown>
#7 0x00000376facd <unknown>
#8 0x00000376fde8 <unknown>
#9 0x000003770486 <unknown>
#10 0x000003725fd9 <unknown>
#11 0x00000376f7f7 <unknown>
#12 0x000003777dde <unknown>
#13 0x000000d3a5a5 <unknown>
#14 0x000000d3a753 <unknown>
#15 0x00000377bd47 <unknown>
#16 0x00000377a383 <unknown>
#17 0x7fc748e13ef7 start_thread
#18 0x7fc74720622f __GI___clone

fish: “./pixelator” terminated by signal SIGABRT (Abort)

Luckily, there is a stub package we can use as a work around–

* nixpkgs.libudev0-shim (libudev0-shim)
    Shim to preserve libudev.so.0 compatibility

How to use libudev0-shim

The trick is to modify your built binary so that its LD_LIBRARY_PATH includes libudev0-shim. This is done via makeWrapper, which wraps the binary with the customized path.

Here’s an example:

with import <nixpkgs> {};

let
  autopatchelf = import ../default.nix;
in
stdenv.mkDerivation {
  name = "pixelator";
  src = ~/downloads/pixelator-1.0.5-linux-x64.zip;
  nativeBuildInputs = [
    libudev0-shim
    udev
    dpkg
    autopatchelf
    unzip
    zlib
    harfbuzz
    freetype
    makeWrapper
];
  libs = stdenv.lib.makeLibraryPath [
    glibc
    libgcc
    gcc-unwrapped
    gtk2-x11
    zlib
    gnome2.pango
    atk
    cairo
    gdk_pixbuf
    glib
    fontconfig
    freetype
    dbus_daemon.lib
    dbus.lib
    xlibs.libX11
    xlibs.libxcb
    xlibs.libXi
    xlibs.libXcursor
    xlibs.libXdamage
    xlibs.libXrandr
    xlibs.libXcomposite
    xlibs.libXext
    nix
    xlibs.libXfixes
    xlibs.libXrender
    xlibs.libXtst
    xlibs.libXScrnSaver
    gnome2.GConf.out
    nss.out
    alsaLib.out
    nspr.out
    cups.lib
    xorg_sys_opengl.out
    expat.out
    harfbuzz.out
    libudev0-shim
];
  installPhase = ''
    chmod +xX pixelator
    chmod +xX _pixelator_cmd.exe
    mkdir -p $out/bin
    mv * $out
    autopatchelf $out

    ln -s $out/pixelator $out/bin/pixelator
    ln -s $out/_pixelator_cmd.exe $out/bin/_pixelator_cmd.exe
  '';
  preFixup = let
    runtimeLibs = lib.makeLibraryPath [ libudev0-shim ];
  in ''
  wrapProgram "$out/pixelator" --prefix LD_LIBRARY_PATH : ${runtimeLibs}
  '';
  dontStrip = true;
}

The highlighted bits are what you need. Notice the wrapping is done during the preFixup phase build phase, so after the binary has been built etc.

How to Install Pixelator on NixOS

Wed, 09 Oct 2019 15:57:00 -0500

Download Pixelator.
Clone my fork of nix-patchtools.
Modify examples/pixelator.nix with the appropriate path to your pixelator-1.0.5-linux-x64.zip.

with import <nixpkgs> {};

let
  autopatchelf = import ../default.nix;
in
stdenv.mkDerivation {
  name = "pixelator";
  src = ~/downloads/pixelator-1.0.5-linux-x64.zip;
  nativeBuildInputs = [
    libudev0-shim
    udev
    dpkg
    autopatchelf
    unzip
    zlib
    harfbuzz
    freetype
    makeWrapper
];
  libs = stdenv.lib.makeLibraryPath [
    glibc
    libgcc
    gcc-unwrapped
    gtk2-x11
    zlib
    gnome2.pango
    atk
    cairo
    gdk_pixbuf
    glib
    fontconfig
    freetype
    dbus_daemon.lib
    dbus.lib
    xlibs.libX11
    xlibs.libxcb
    xlibs.libXi
    xlibs.libXcursor
    xlibs.libXdamage
    xlibs.libXrandr
    xlibs.libXcomposite
    xlibs.libXext
    nix
    xlibs.libXfixes
    xlibs.libXrender
    xlibs.libXtst
    xlibs.libXScrnSaver
    gnome2.GConf.out
    nss.out
    alsaLib.out
    nspr.out
    cups.lib
    xorg_sys_opengl.out
    expat.out
    harfbuzz.out
    libudev0-shim
];
  installPhase = ''
    chmod +xX pixelator
    chmod +xX _pixelator_cmd.exe
    mkdir -p $out/bin
    mv * $out
    autopatchelf $out

    ln -s $out/pixelator $out/bin/pixelator
    ln -s $out/_pixelator_cmd.exe $out/bin/_pixelator_cmd.exe
  '';
  preFixup = let
    runtimeLibs = lib.makeLibraryPath [ libudev0-shim ];
  in ''
  wrapProgram "$out/pixelator" --prefix LD_LIBRARY_PATH : ${runtimeLibs}
  '';
  dontStrip = true;
}

Building it

$ cd nix-patchtools/examples
$ nix-build pixelator.nix

The output will be something like,

/nix/store/j89yqn3zhpz2r9jwrng45yl09arxq4h0-pixelator

Installing it

nix-env -i /nix/store/j89yqn3zhpz2r9jwrng45yl09arxq4h0-pixelator

Running it

Pixelator expects the pixelator and _pixelator_cmd.exe binaries to be in the directory that you launch it from. You can solve that via

$ mkdir -p ~/scratch/pixelator
$ cd ~/scratch/pixelator
$ ln -s /nix/store/ksxwcam55sl20bpv39lj1244n5kvcnq9-pixelator/_pixelator_cmd.exe .
$ ln -s /nix/store/ksxwcam55sl20bpv39lj1244n5kvcnq9-pixelator/pixelator pixelator

Success!

How to Look up a URL's PageRank with Ruby + Open PageRank's API

Thu, 03 Oct 2019 14:24:00 -0500

DomCom’s Open PageRank is a free alternative to Google’s PageRank and includes a public API. All you gotta do before you can use it is fill out this form and they’ll email you an access key.

The limits are generous:

Based on our current limits you can get the PageRank data for 4.3 million domains every single day for a single API Key. If you need more data simply contact us and we will increase the limit.

4.3 million… yeah, that should about cover it.

Using it

The example in their API docs is written in PHP. I’ve translated it here into Ruby, took a bit of fiddling.

require 'net/http'
require 'active_support/all'

uri = URI.parse("https://openpagerank.com/api/v1.0/getPageRank")
uri.query = ["rs.io", "example.com"].to_query('domains')

req = Net::HTTP::Get.new(uri)
req['API-OPR'] = YOUR_SECRET_KEY

res = Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |http|
  http.request(req)
}

result = JSON.parse(res.body)
puts JSON.pretty_generate(result)

Swap YOUR_SECRET_KEY for your actual key and replace the array ["rs.io", "example.com"] with whatever URLs you want to look up, max 100 in a single request like this.

You should get something like,

{
  "status_code": 200,
  "response": [
    {
      "status_code": 200,
      "error": "",
      "page_rank_integer": 4,
      "page_rank_decimal": 4.32,
      "rank": "1443770",
      "domain": "rs.io"
    },
    {
      "status_code": 200,
      "error": "",
      "page_rank_integer": 8,
      "page_rank_decimal": 7.88,
      "rank": "193",
      "domain": "example.com"
    }
  ],
  "last_updated": "16th Aug 2019"
}

The metrics are calculated from Common Crawl’s “open repository of web crawl data that can be accessed and analyzed by anyone”–very cool!

Black progress

Wed, 02 Oct 2019 16:25:00 -0500

From 2001 to 2017, the incarceration rate for black men declined by 34 percent.

Between 2001 and 2017, the birth rate for black women aged 15–19 declined by 63 percent. In fact, the black teenage birth rate in 2017 was lower than the white teenage rate as recently as 2002.

In 2017, black female life expectancy was 78.5 years, up from 75.1 years in 2000. Life expectancy for black men increased from 68.2 to 71.9 years over the same timespan.

2018 census data showed that 37 percent of black Americans aged 25–34 had some kind of college degree. If black America were its own country, that would place it in between Germany (31 percent) and Spain (43 percent) in terms of educational attainment. What’s more, the economist Raj Chetty has found that black women, though less likely to attend college than white women, are now more likely to attend college than white men from similar socioeconomic backgrounds.

The Federal Reserve recently reported that over 60 percent of blacks at every level of educational attainment say they’re doing better financially than their parents—a higher percentage than either whites or Hispanics.

Chetty has found that black women now go on to earn slightly higher incomes than white women from similar socioeconomic backgrounds.

–from The Case for Black Optimism

Tall trees

Wed, 02 Oct 2019 16:09:00 -0500

Researchers Discover the Tallest Known Tree in the Amazon

88.5 meters, or over 290 feet

This tree is not as tall as the tallest tree in the bay area! There is a 328-foot redwood tree in Big Basin.

Amazing a tree that tall can get things from the root that high. Unless the tops work differently?

Redwoods depend on the humidity and fog from the pacific coast to get water to its highest parts and to keep it from drying out.

Foliar Uptake of Fog in the Coast Redwood Ecosystem: a Novel Drought-Alleviation Strategy Shared by Most Redwood Forest Plant

As hedge fund grandma used to say