Blog
1. January 2020
Can the dynamic linking of neural activations bring us closer to strong AI?
Most artificial neural networks ignore the spiking nature of biological neural networks to simplify the underlying model and enable learning techniques such as back-propagation. But by doing so, aren’t we possibly rejecting one of the most central principles of biological neural networks?
Within most artificial neural network models, an activation is just a real numbered value associated with the neuron itself. But that’s not what is happening within a biological neural network. Here, an activation occurs when the threshold of a neuron is exceeded and there is an exact point in time associated with it. A prerequisite for this event to occur is that several other input activations have been fired before the current activation.
...
23. September 2019
Using Information Gain for the Unsupervised Training of Excitatory Neurons
Looking for a biologically more plausible way to train a neural network.
Traditionally, artificial neural networks have been trained using the Delta rule and backpropagation. But this contradicts the findings that the neurosciences have made on the function of the brain. There simply is no gradient error signal that is propagated backwards through biological neurons.
...
28. August 2019
Using Meta-Neurons to learn facts from a single training example
When people see a new animal, meet a new person or visit a new place, they don’t need to repeat that experience thousands of times to remember it — so why should computers have to?
Human learning comes in two forms, a fast and a slow one. The slow one requires a lot of repetition which seems to be necessary to conquer a new cognitive field such as learning a new language. But once a field is mastered, learning new facts within this field requires very few, possibly even only one example. It appears, that the brain regions involved in processing this field have been pre wired to the regions they depend on. So once a new fact needs to be learned, this pre wiring is used to speed up the training of the neurons involved in processing this new fact. It is important, that during this training a distinction is made between new and existing knowledge. Otherwise, all connections based on already existing knowledge would be lost due to the repeated learning of something that is already known. So the neurons involved in the processing of an already known fact must therefore be able to suppress the formation of new knowledge.
...
9. August 2019
How to efficiently propagate activations in a massive neural network
When people see a new animal, meet a new person or visit a new place, they don’t need to repeat that experience thousands of times to remember it — so why should computers have to?
In traditional neural networks using the sigmoid activation function, all neurons are more or less activated. There is no clear case of an inactive neuron here.
...
3. August 2019
On adding negative feedback synapses to a neural network
The missing link in deep neural networks
The special thing about adding negative recurrent synapses to a neural network is that they introduce inner states within the network.
...
9. December 2018
On integrating symbolic inference into deep neural networks
What makes biological neural networks so superior to their technical counterparts? Is there anything we have overlooked so far?
Deep neural networks have been a tremendous success story over the last couple of years. Many advances in the field of AI, such as recognizing real world objects, fluently translating natural language or playing GO ...
25. October 2017
Aika: Ein semantisches neuronales Netzwerk
Wenn es darum geht Informationen aus natürlichsprachigen Texten zu extrahieren, stehen einem verschiedene Möglichkeiten zur Verfügung. Eine der ältesten und wohl auch am häufigsten genutzten Möglichkeiten ist die der regulären Ausdrücke. Hier werden exakte Muster definiert und in einem Textstring gematcht. Probleme bereiten diese allerdings, wenn kompliziertere semantische Muster gefunden werden sollen oder wenn verschiedene Muster aufeinander aufbauen oder miteinander interagieren sollen. ...
20. June 2017
Machine-Learning-Bibliothek hilft dabei Texte zu verstehen
Volltextsuche ist ein Kernbestandteil des Internetzeitalters, nichtsdestotrotz lässt sie bis heute viel zu Wünschen übrig. Sie ist ganz hervorragend dazu geeignet, um exakte Worttreffer in einer großen Menge an Dokumenten zu finden. Was allerdings bis heute noch nicht zuverlässig funktioniert ist, ein Wort nur in seiner gewünschten Bedeutung zu finden. Genau hier kann die Java-Bibliothek Aika weiterhelfen. ...