Naveen Venkat

DLCV – PyTorch Tutorial

#ServerProcessor — Tue, 11 Feb 2020 14:27:41 +0000

Here is the Colab Notebook used in the class.

Resources for some Stanford’s AI Courses

#ServerProcessor — Wed, 18 Dec 2019 04:37:04 +0000

Here is a list of courses that are a part of Stanford’s AI Graduate Certificate, along with the resources (Slides / Notes / Lecture Videos etc.) for the courses.

Courses. (Required: CS221 and 3 elective courses from the rest)

4 units. CS221, CS224N, CS224U, CS228, CS229, CS230, CS231A, CS231N, AA228
3 units. CS157, CS223A, CS234, CS236, CS330

Prerequisites.

CS109 Probability for Computer Scientists Slides & Notes
Math104 Applied Matrix Theory Notes / HW

Course Materials

CS221 Artificial Intelligence: Principles and Techniques Materials
CS224N Natural Language Processing with Deep Learning YouTube / Other materials
CS224U Natural Language Understanding Materials
CS228 Probabilistic Graphical Models: Principles and Techniques Official Notes
CS229 Machine Learning Coursera / Official Notes
CS230 Deep Learning Slides & Materials / Coursera
CS231A Computer Vision: From 3D Reconstruction to Recognition Official Notes
CS231N Convolutional Neural Networks for Visual Recognition Youtube / Slides & Code
AA228 Decision Making Under Uncertainty Reading / HW Exercises / Other materials
CS157 Computational Logic Github Notes
CS223A Introduction to Robotics All Materials
CS234 Reinforcement Learning Slides & Notes / Lecture Videos
CS236 Deep Generative Models Notes / Slides & HW
CS330 Deep Multi-task and Meta Learning Notes, Slides & HW

Deep Learning for Computer Vision – A Beginner’s Checklist

#ServerProcessor — Sun, 01 Dec 2019 07:53:05 +0000

This document contains a list of resources to learn theoretical concepts in Machine Learning and to practice writing code online.

Download PDF (64 KB)

Uniform sampling within an ellipsoid

#ServerProcessor — Thu, 28 Feb 2019 09:30:33 +0000

Written for PyTorch with CUDA compatibility.

Use Case: Instead of doing a multivariate normal sampling (available in torch.distributions.multivariate_normal), one could also do a random sampling within a specified confidence region of the multivariate gaussian function.

Although it will be an approximation, one could obtain a confidence region using the variance across each dimension (diagonal of the covariance matrix). Suppose we define a 3*s boundary (Upto 3 x variance across each dimension is an accepted confidence interval), then EllipsoidSampler can be used to construct such an ellipsoid (with given mean = mu and lengths of axes = axes. This utility will further help sample from within the ellipsoid in a random fashion (instead of a random normal fashion).

Get code on gist.github

Entropy, Cross-Entropy & KL-Divergence

#ServerProcessor — Sun, 28 Oct 2018 15:13:46 +0000

Here is a 10-minute video by Aurélien Géron explaining entropy, cross-entropy and KL-divergence using Information Theory.

He has some more interesting videos on his channel. Do check it out!

Connect-3 Game Playing Bot

#ServerProcessor — Mon, 15 Oct 2018 19:17:18 +0000

We use Minimax algorithm to predict the next optimal move after every move by the user. This work demonstrates how a complete search done by Minimax algorithm can always yield optimal results. To speed up the search, alpha-beta prunning is implemented to prune moves that do no better than the currently explored moves. We test two methods using Minimax algorithm for game playing – with and without Alpha-Beta Prunning. A 14x faster first move is obtained using alpha-beta prunning.

The code is publicly available on github.

Probabilistic Inference using Bayesian Networks

#ServerProcessor — Mon, 15 Oct 2018 19:09:59 +0000

In this work, we study the application of bayesian networks for probabilistic inference. We consider a hypothetical real-world scenario where we answer queries regarding various events (health problems, accidents etc.) caused by factors such as air pollution, bad road conditions etc.

Each event/factor is modeled as a random variable with a certain probability distribution function (given as input). Variable dependence graph is constructed and bayes rule is applied on the markov blanket of the query variables to reduce the computational effort. Detailed documentation can be found in the code.

The code is publicly available at github.

Visualizing Perceptron Learning

#ServerProcessor — Mon, 15 Oct 2018 19:05:06 +0000

This program visualizes the learning process of a perceptron. For simplicity, we consider the perceptron to learn the identity function. We give a 2 dimensional input and classify each point as being below the line or above the line (binary classification). We update the weights of the perceptron whenever misclassification occurs. Over several examples, the perceptron learns the identity mapping.

The code is publicly available on github.

The Curse of Dimensionality: Inside Out

#ServerProcessor — Mon, 17 Sep 2018 05:37:45 +0000

The Curse of Dimensionality, introduced by Bellman, refers to the explosive nature of spatial dimensions and its resulting effects, such as, an exponential increase in computational effort, large waste of space and poor visualization capabilities. Higher number of dimensions theoretically allow more information to be stored, but practically rarely help due to the higher possibility of noise and redundancy in real world data. In this article, the effects of high dimensionality is studied through various experiments and the possible solutions to counter or mitigate such effects are proposed. The source code of the experiments performed is available publicly on github.

Read the Paper | Get the Code

Building an inline Turing Machine for C++

#ServerProcessor — Wed, 29 Aug 2018 13:41:36 +0000

Introduction

Turing Machines are mathematical abstractions of a computing systems. Suppose you are given an input string of letters of a given alphabet, and you want to do some computation on it, then given enough time and memory, a Turing Machine can do the job for you. The only caveat is, the language that you want to compute upon should be “computable”. What this means is, there are certain languages that are not computable in reasonable time by a Turing Machine. Such languages are called “hard”, because it is hard to write an algorithm to compute in reasonable time. This is a very abstract version of what a Turing Machine is. Let us explore it in brief.

Turing Machines in Brief

In a Turing Machine, we define an input tape, which is some form of a memory device. Consider the cassette tapes that were introduced around the 1960s. We call such a memory model as a sequential linear memory, because:

sequential: to reach a specific place in the tape, you have to go through all the locations from the source to the destination.
linear: because the traversal is only in one “linear” dimension.

Now, given this memory unit, we need a computing unit that could perform operations on this memory. We achieve this by defining a computing machine that has the capability of:

traversing the memory
reading from the memory
writing to the memory
make decisions based on some algorithm / set of rules

Given these four abilities (or operations), our machine will be able to compute anything (almost!).

Defining the Turing Machine in C++

The infinite tape is handled by allocating an arbitrarily large array of integers. Simply put, we use the array as an abstract version of the infinite tape, and reallocate memory as required. For this demonstration, we fix a length for the array (buffer size), and point to the current cell using a data pointer variable (DP).

Traversing the memory: We can only move left or right on the tape. Thus we define two functions _L() and _R() that decrease and increase the DP. We map the function calls to the macros L and R to avoid repeated parenthesis.
Reading the memory: Accessing the memory can be simply done by de-referencing the data pointer (*DP). This is equivalent to the current data value (CDV). We also define a macro O to output the CDV.
Writing to the memory: We can modify the memory by either incrementing / decrementing the current cell using macros U and D respectively, or take an input directly into the cell using I.
Make decisions based on rules: To have a control structure, we define a LOOP macro that simply loops the block of code following it, until the data pointed by the DP is 0 at the beginning of an iteration.

These functions are summarized in the following table:

L	Move the pointer to the left (Left)
R	Move the pointer to the right (Right)
U	Increment the current pointed cell by 1 (Up)
D	Decrement the current pointer cell by 1 (Down)
I	Read a single character into the current cell (Input)
O	Print the character at the current cell (Output)
LOOP	The block following it will be looped until a 0 occurs at the current cell when loop condition is checked

Ability to perform complex algorithms

Since the framework is written in C++, we can leverage the function definition capabilities of the language. In version 0.2, we also implement a macro TM that let’s us define a void function as a turing machine that works directly on the tape. Since the tape is global, we do not worry about the inputs and the outputs of the function, and keep empty parameters. Some examples can be found here.

Typical Use-Cases

This framework lets us run an inline Turing Machine in the C++ code, adding meta coding capabilities. This Turing Machine can be used as an auxiliary memory. Some examples can be, using it as an inline version control system, using it for policy agnostic programming, or even writing complex Turing Machines in a modular design.

Getting Started

The project is availible on github. Modular examples can be found here.

Cite:

Naveen Venkat (2016). Building an Inline Turing Machine for C++. Retrieved from http://www.naveenvenkat.com