pax_global_header 0000666 0000000 0000000 00000000064 13366231667 0014527 g ustar 00root root 0000000 0000000 52 comment=2f7765add97abd49e64861b218e5e6b7d0a5c622
aika-java-0.22/ 0000775 0000000 0000000 00000000000 13366231667 0013276 5 ustar 00root root 0000000 0000000 aika-java-0.22/.gitignore 0000664 0000000 0000000 00000000324 13366231667 0015265 0 ustar 00root root 0000000 0000000 # Maven build files
target
*.log
maven-eclipse.xml
build.properties
site-content
*~
# IntelliJ IDEA files
.idea
.iws
*.iml
*.ipr
# Eclipse files
.settings
.classpath
.project
.externalToolBuilders
.checkstyle
aika-java-0.22/LICENSE.txt 0000664 0000000 0000000 00000026134 13366231667 0015127 0 ustar 00root root 0000000 0000000
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2015-2017 Lukas Molzberger
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
aika-java-0.22/NOTICE.txt 0000664 0000000 0000000 00000000053 13366231667 0015016 0 ustar 00root root 0000000 0000000 AIKA
Copyright 2015-2017 Lukas Molzberger
aika-java-0.22/README.md 0000664 0000000 0000000 00000000707 13366231667 0014561 0 ustar 00root root 0000000 0000000 # aika
A good starting point to get familiar with this project are probably the following three test cases:
- SimplePatternMatchingTest (Demonstrates how a pattern like a word or a phrase can be matched)
- MutualExclusionTest (Demonstrates a negative feed back loop)
- NamedEntityRecognitionTest (A more complex example: recognize the name jackson cook)
- SuspensionTest (Demonstrates how neurons and logic nodes can be suspended to an external storage) aika-java-0.22/pom.xml 0000664 0000000 0000000 00000016776 13366231667 0014634 0 ustar 00root root 0000000 0000000 4.0.0org.aika-softwareaikajar0.22aikahttp://aika.networkAn artificial intelligence for knowledge acquisitionLukas MolzbergerLukas Molzbergercontact@aika-software.orgApache License, Version 2.0http://www.apache.org/licenses/LICENSE-2.0.txtrepohttps://github.com/aika-algorithm/aikascm:git:git://github.com/aika-algorithm/aika.gitscm:git:git@github.com:aika-algorithm/aika.gitaika-0.22ossrhhttps://oss.sonatype.org/content/repositories/snapshotsossrhhttps://oss.sonatype.org/service/local/staging/deploy/maven2/4.123.6.1org.apache.maven.pluginsmaven-compiler-plugin3.31.81.8utf-8org.apache.maven.pluginsmaven-surefire-plugin2.18.1always-Xmx8000m**/*Test.javaorg.apache.maven.pluginsmaven-javadoc-plugin2.10.3attach-javadocsjar-Xdoclint:noneorg.apache.maven.pluginsmaven-source-plugin2.2.1attach-sourcesjar-no-forkfalse../src/main/resources**/*false../src/test/resources**/*org.apache.commonscommons-math3${commons-math.version}org.slf4jslf4j-api1.7.25ch.qos.logbacklogback-core1.2.3testch.qos.logbacklogback-classic1.2.3testorg.slf4jjcl-over-slf4j1.7.25org.slf4jjul-to-slf4j1.7.25junitjunit${junit.version}testcommons-iocommons-io2.5testdeploy_artifactorg.apache.maven.pluginsmaven-gpg-plugin1.5sign-artifactsverifysignorg.sonatype.pluginsnexus-staging-maven-plugin1.6.7trueossrhhttps://oss.sonatype.org/true
aika-java-0.22/src/ 0000775 0000000 0000000 00000000000 13366231667 0014065 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/ 0000775 0000000 0000000 00000000000 13366231667 0015011 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/java/ 0000775 0000000 0000000 00000000000 13366231667 0015732 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/java/network/ 0000775 0000000 0000000 00000000000 13366231667 0017423 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/java/network/aika/ 0000775 0000000 0000000 00000000000 13366231667 0020330 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/java/network/aika/AbstractNode.java 0000664 0000000 0000000 00000003603 13366231667 0023546 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import network.aika.lattice.Node;
import network.aika.lattice.NodeActivation;
import network.aika.neuron.INeuron;
import network.aika.neuron.Neuron;
import java.io.DataInput;
import java.io.IOException;
/**
*
* @author Lukas Molzberger
*/
public abstract class AbstractNode
, A extends NodeActivation> implements Writable {
public volatile int lastUsedDocumentId = 0;
public volatile boolean modified;
public P provider;
/**
* Propagate an activation to the next node or the next neuron that is depending on the current node.
*
* @param act
*/
public abstract void propagate(A act);
public void setModified() {
modified = true;
}
public void suspend() {}
public void reactivate() {}
public static
AbstractNode read(DataInput in, P p) throws IOException {
AbstractNode n;
if(in.readBoolean()) {
n = INeuron.readNeuron(in, (Neuron) p);
} else {
n = Node.readNode(in, p);
}
return n;
}
}
aika-java-0.22/src/main/java/network/aika/ActivationFunction.java 0000664 0000000 0000000 00000001043 13366231667 0025000 0 ustar 00root root 0000000 0000000 package network.aika;
public enum ActivationFunction {
RECTIFIED_SCALED_LOGISTIC_SIGMOID(x -> Math.max(0.0, (2.0 * Utils.sigmoid(x)) - 1.0)),
RECTIFIED_HYPERBOLIC_TANGENT(x -> Math.max(0.0, Math.tanh(x))),
RECTIFIED_LINEAR_UNIT(x -> Math.max(0.0, x)),
LIMITED_RECTIFIED_LINEAR_UNIT(x -> Math.max(0.0, Math.min(1.0, x)));
Function f;
ActivationFunction(Function f) {
this.f = f;
}
public double f(double x) {
return f.f(x);
}
interface Function {
double f(double x);
}
}
aika-java-0.22/src/main/java/network/aika/Converter.java 0000664 0000000 0000000 00000026566 13366231667 0023161 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import network.aika.lattice.AndNode;
import network.aika.lattice.Node;
import network.aika.lattice.OrNode;
import network.aika.neuron.INeuron;
import network.aika.neuron.relation.Relation;
import network.aika.neuron.Synapse;
import java.util.*;
import static network.aika.neuron.range.Range.Mapping.NONE;
/**
* Converts the synapse weights of a neuron into a boolean logic representation of this neuron.
*
* @author Lukas Molzberger
*/
public class Converter {
public static int MAX_AND_NODE_SIZE = 6;
public static Comparator SYNAPSE_COMP = (s1, s2) -> {
int r = Boolean.compare(
s2.rangeOutput.begin != NONE || s2.rangeOutput.end != NONE || s2.identity,
s1.rangeOutput.begin != NONE || s1.rangeOutput.end != NONE || s1.identity
);
if (r != 0) return r;
r = Double.compare(s2.weight, s1.weight);
if (r != 0) return r;
return Integer.compare(s1.id, s2.id);
};
private int threadId;
private INeuron neuron;
private Document doc;
private OrNode outputNode;
private Collection modifiedSynapses;
public static boolean convert(int threadId, Document doc, INeuron neuron, Collection modifiedSynapses) {
return new Converter(threadId, doc, neuron, modifiedSynapses).convert();
}
private Converter(int threadId, Document doc, INeuron neuron, Collection modifiedSynapses) {
this.doc = doc;
this.neuron = neuron;
this.threadId = threadId;
this.modifiedSynapses = modifiedSynapses;
}
private boolean convert() {
outputNode = neuron.node.get();
initInputNodesAndComputeWeightSums();
initCreateBeginEndPositionFlags();
if(neuron.biasSum + neuron.posDirSum + neuron.posRecSum <= 0.0) {
neuron.requiredSum = neuron.posDirSum + neuron.posRecSum;
outputNode.removeParents(threadId);
return false;
}
List candidates = prepareCandidates();
NodeContext nodeContext = null;
boolean noFurtherRefinement = false;
TreeSet reqSyns = new TreeSet<>(Synapse.INPUT_SYNAPSE_COMP);
double sum = 0.0;
neuron.requiredSum = 0.0;
if(neuron.numDisjunctiveSynapses == 0) {
double remainingSum = neuron.posDirSum;
int i = 0;
for (Synapse s : candidates) {
double v = s.getMaxInputValue();
final boolean isOptionalInput = sum + remainingSum - v + neuron.posRecSum + neuron.posPassiveSum + neuron.biasSum > 0.0;
final boolean maxAndNodesReached = i >= MAX_AND_NODE_SIZE;
if (isOptionalInput || maxAndNodesReached) {
break;
}
remainingSum -= v;
neuron.requiredSum += v;
reqSyns.add(s);
NodeContext nlNodeContext = expandNode(nodeContext, s);
if(nlNodeContext == null) {
break;
}
nodeContext = nlNodeContext;
i++;
sum += v;
final boolean sumOfSynapseWeightsAboveThreshold = sum + neuron.posRecSum + neuron.posPassiveSum + neuron.biasSum > 0.0;
if (sumOfSynapseWeightsAboveThreshold) {
noFurtherRefinement = true;
break;
}
}
outputNode.removeParents(threadId);
if (noFurtherRefinement || i == MAX_AND_NODE_SIZE) {
outputNode.addInput(nodeContext.getSynapseIds(), threadId, nodeContext.node, true);
} else {
for (Synapse s : candidates) {
double v = s.getMaxInputValue();
boolean belowThreshold = sum + v + remainingSum + neuron.posRecSum + neuron.posPassiveSum + neuron.biasSum <= 0.0;
if (belowThreshold) {
break;
}
if (!reqSyns.contains(s)) {
NodeContext nlNodeContext = expandNode(nodeContext, s);
if(nlNodeContext != null) {
outputNode.addInput(nlNodeContext.getSynapseIds(), threadId, nlNodeContext.node, true);
remainingSum -= v;
}
}
}
}
} else {
for (Synapse s : modifiedSynapses) {
if (s.getMaxInputValue() + neuron.posRecSum + neuron.posPassiveSum + neuron.biasSum > 0.0) {
NodeContext nlNodeContext = expandNode(nodeContext, s);
outputNode.addInput(nlNodeContext.getSynapseIds(), threadId, nlNodeContext.node, false);
}
}
}
return true;
}
private void initCreateBeginEndPositionFlags() {
modifiedSynapses.forEach(s -> {
if (s.rangeOutput.begin != NONE) {
neuron.createBeginPosition = false;
}
if (s.rangeOutput.end != NONE) {
neuron.createEndPosition = false;
}
});
}
private List prepareCandidates() {
Synapse syn = getBestSynapse(neuron.inputSynapses.values());
TreeSet alreadyCollected = new TreeSet<>();
ArrayList selectedCandidates = new ArrayList<>();
TreeMap relatedSyns = new TreeMap<>();
while(syn != null && selectedCandidates.size() < MAX_AND_NODE_SIZE) {
relatedSyns.remove(syn.id);
selectedCandidates.add(syn);
alreadyCollected.add(syn.id);
for(Integer relId: syn.relations.keySet()) {
if(!alreadyCollected.contains(relId)) {
Synapse rs = syn.output.getSynapseById(relId);
if(rs != null) {
relatedSyns.put(relId, rs);
}
}
}
syn = getBestSynapse(relatedSyns.values());
}
return selectedCandidates;
}
private Synapse getBestSynapse(Collection synapses) {
Synapse maxSyn = null;
for(Synapse s: synapses) {
if(!s.isNegative() && !s.isRecurrent && !s.inactive && !s.input.get().isPassiveInputNeuron()) {
if(maxSyn == null || SYNAPSE_COMP.compare(maxSyn, s) > 0) {
maxSyn = s;
}
}
}
return maxSyn;
}
public static final int DIRECT = 0;
public static final int RECURRENT = 1;
public static final int POSITIVE = 0;
public static final int NEGATIVE = 1;
private void initInputNodesAndComputeWeightSums() {
double[][] sumDelta = new double[2][2];
double posPassiveSumDelta = 0.0;
for (Synapse s : modifiedSynapses) {
if(s.toBeDeleted) {
s.update(doc, -s.weight, 0.0, s.limit);
}
INeuron in = s.input.get();
in.lock.acquireWriteLock();
try {
if (!s.inactive) {
sumDelta[s.isRecurrent ? RECURRENT : DIRECT][s.isNegative() ? NEGATIVE : POSITIVE] -= s.limit * s.weight;
sumDelta[s.isRecurrent ? RECURRENT : DIRECT][s.getNewWeight() <= 0.0 ? NEGATIVE : POSITIVE] += (s.limit + s.limitDelta) * s.getNewWeight();
if(in.isPassiveInputNeuron() && !s.isNegative()) {
posPassiveSumDelta -= !s.isNegative() ? (s.limit * s.weight) : 0.0;
posPassiveSumDelta += s.getNewWeight() > 0.0 ? ((s.limit + s.limitDelta) * s.getNewWeight()) : 0.0;
}
if (s.isConjunction(false, true) && !s.isConjunction(true, true)) {
neuron.numDisjunctiveSynapses++;
}
}
s.weight += s.weightDelta;
s.weightDelta = 0.0;
s.bias += s.biasDelta;
s.biasDelta = 0.0;
s.limit += s.limitDelta;
s.limitDelta = 0.0;
if (doc != null) {
s.committedInDoc = doc.id;
}
} finally {
in.lock.releaseWriteLock();
}
if(s.toBeDeleted) {
s.unlink();
}
}
neuron.bias += neuron.biasDelta;
neuron.biasDelta = 0.0;
neuron.biasSum += neuron.biasSumDelta;
neuron.biasSumDelta = 0.0;
assert Double.isFinite(neuron.biasSum);
neuron.posDirSum += sumDelta[DIRECT][POSITIVE];
neuron.negDirSum += sumDelta[DIRECT][NEGATIVE];
neuron.negRecSum += sumDelta[RECURRENT][NEGATIVE];
neuron.posRecSum += sumDelta[RECURRENT][POSITIVE];
neuron.posPassiveSum += posPassiveSumDelta;
neuron.setModified();
}
private NodeContext expandNode(NodeContext nc, Synapse s) {
if (nc == null) {
NodeContext nln = new NodeContext();
nln.node = s.input.get().outputNode.get();
nln.offsets = new Synapse[] {s};
return nln;
} else {
Relation[] relations = new Relation[nc.offsets.length];
for(int i = 0; i < nc.offsets.length; i++) {
Synapse linkedSynapse = nc.offsets[i];
Set relSet = s.getRelationById(linkedSynapse.id);
if (relSet != null) {
assert relSet.size() == 1;
relations[i] = relSet.iterator().next();
}
}
NodeContext nln = new NodeContext();
nln.offsets = new Synapse[nc.offsets.length + 1];
AndNode.Refinement ref = new AndNode.Refinement(new AndNode.RelationsMap(relations), s.input.get().outputNode);
AndNode.RefValue rv = nc.node.extend(threadId, doc, ref, null);
if(rv == null) {
return null;
}
nln.node = rv.child.get(doc);
for (int i = 0; i < nc.offsets.length; i++) {
nln.offsets[rv.offsets[i]] = nc.offsets[i];
}
for (int i = 0; i < nln.offsets.length; i++) {
if (nln.offsets[i] == null) {
nln.offsets[i] = s;
}
}
return nln;
}
}
private class NodeContext {
Node node;
Synapse[] offsets;
int[] getSynapseIds() {
int[] result = new int[offsets.length];
for(int i = 0; i < result.length; i++) {
result[i] = offsets[i].id;
}
return result;
}
}
}
aika-java-0.22/src/main/java/network/aika/DistanceFunction.java 0000664 0000000 0000000 00000001000 13366231667 0024422 0 ustar 00root root 0000000 0000000 package network.aika;
import network.aika.neuron.activation.Activation;
public enum DistanceFunction {
NONE((iAct, oAct) -> 1.0),
DEGRADING((iAct, oAct) -> 1.0 / ((double) (1 + Math.abs(iAct.range.begin.getDistance(oAct.range.begin)))));
Function f;
DistanceFunction(Function f) {
this.f = f;
}
public double f(Activation iAct, Activation oAct) {
return f.f(iAct, oAct);
}
interface Function {
double f(Activation iAct, Activation oAct);
}
}
aika-java-0.22/src/main/java/network/aika/Document.java 0000664 0000000 0000000 00000051733 13366231667 0022762 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import network.aika.lattice.Node;
import network.aika.lattice.NodeActivation;
import network.aika.neuron.INeuron;
import network.aika.neuron.Synapse;
import network.aika.neuron.activation.Activation;
import network.aika.neuron.activation.Activation.Link;
import network.aika.neuron.activation.Candidate;
import network.aika.neuron.range.Position;
import network.aika.neuron.range.Range;
import network.aika.neuron.activation.SearchNode;
import network.aika.lattice.Node.ThreadState;
import network.aika.neuron.activation.*;
import network.aika.neuron.activation.SearchNode.Decision;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static network.aika.neuron.activation.SearchNode.Decision.UNKNOWN;
/**
* The {@code Document} class represents a single document which may be either used for processing a text or as
* training input. A document consists of the raw text, the interpretations and the activations.
*
*
When the document is not needed any more, the method {@code clearActivations} must be called, since Aika only
* supports a single document per thread and model.
*
* @author Lukas Molzberger
*/
public class Document implements Comparable {
private static final Logger log = LoggerFactory.getLogger(Document.class);
public static int CLEANUP_INTERVAL = 500;
public static int MAX_ROUND = 20;
public static int ROUND_LIMIT = -1;
/**
* Experimental code: not working yet!
*/
public static boolean INCREMENTAL_MODE = false;
public final int id;
private final StringBuilder content;
public long visitedCounter = 1;
public int activationIdCounter = 0;
public int logicNodeActivationIdCounter = 0;
public int searchNodeIdCounter = 0;
public int searchStepCounter = 0;
public int positionIdCounter = 0;
public Model model;
public int threadId;
public Queue queue = new Queue();
public ValueQueue vQueue = new ValueQueue();
public UpperBoundQueue ubQueue = new UpperBoundQueue();
public Linker linker;
public TreeMap positions = new TreeMap<>();
public TreeSet activatedNodes = new TreeSet<>();
public TreeSet activatedNeurons = new TreeSet<>();
public TreeSet finallyActivatedNeurons = new TreeSet<>();
public TreeSet inputNeuronActivations = new TreeSet<>();
public TreeMap> modifiedWeights = new TreeMap<>();
public TreeMap searchNodeWeights = new TreeMap<>();
private TreeMap activationsByRangeBegin = new TreeMap<>((ak1, ak2) -> {
int r = Position.compare(ak1.range.begin, ak2.range.begin);
if (r != 0) return r;
r = ak1.node.compareTo(ak2.node);
if (r != 0) return r;
return Integer.compare(ak1.actId, ak2.actId);
});
private TreeMap activationsByRangeEnd = new TreeMap<>((ak1, ak2) -> {
int r = Position.compare(ak1.range.end, ak2.range.end);
if (r != 0) return r;
r = ak1.node.compareTo(ak2.node);
if (r != 0) return r;
return Integer.compare(ak1.actId, ak2.actId);
});
public TreeMap activationsById = new TreeMap<>();
private int lastProcessedActivationId = -1;
private int lastLinkedActivationId = -1;
public static class ActKey {
Range range;
Node node;
int actId;
public ActKey(Range range, Node node, int actId) {
this.range = range;
this.node = node;
this.actId = actId;
}
}
public TreeSet addedNodes = new TreeSet<>();
public ArrayList addedNodeActivations = new ArrayList<>();
public SearchNode selectedSearchNode;
public ArrayList candidates = new ArrayList<>();
public long createV;
public static Comparator ACTIVATIONS_OUTPUT_COMPARATOR = (act1, act2) -> {
int r = Range.compare(act1.range, act2.range, false);
if (r != 0) return r;
r = act1.node.compareTo(act2.node);
if (r != 0) return r;
return Integer.compare(act1.id, act2.id);
};
public Document(int id, String content, Model model, int threadId) {
this.id = id;
this.content = new StringBuilder(content);
this.model = model;
this.threadId = threadId;
this.linker = model.getLinkerFactory().createLinker(this);
}
public void append(String txt) {
content.append(txt);
}
public char charAt(int i) {
return content.charAt(i);
}
public String getContent() {
return content.toString();
}
public int length() {
return content.length();
}
public String toString() {
return content.toString();
}
public Position lookupFinalPosition(int pos) {
Position p = positions.get(pos);
if(p == null) {
p = new Position(this, pos);
positions.put(pos, p);
}
return p;
}
public String getText(Range r) {
if(r.begin.getFinalPosition() != null && r.end.getFinalPosition() != null) {
return content.substring(
Math.max(0, Math.min(r.begin.getFinalPosition(), length())),
Math.max(0, Math.min(r.end.getFinalPosition(), length()))
);
} else {
return "";
}
}
public void addActivation(Activation act) {
ActKey dak = new ActKey(act.range, act.node, act.id);
if (act.range.begin != null && act.range.begin.getFinalPosition() != null) {
activationsByRangeBegin.put(dak, act);
}
if (act.range.end != null && act.range.end.getFinalPosition() != null) {
activationsByRangeEnd.put(dak, act);
}
activationsById.put(act.id, act);
}
public Collection getActivations(boolean onlyFinal) {
if(!onlyFinal) {
return activationsById.values();
} else {
return activationsById
.values()
.stream()
.filter(act -> act.isFinalActivation())
.collect(Collectors.toList());
}
}
public Collection getActivationsByRangeBegin(Range fromKey, boolean fromInclusive, Range toKey, boolean toInclusive) {
return activationsByRangeBegin.subMap(
new Document.ActKey(fromKey, Node.MIN_NODE, Integer.MIN_VALUE),
fromInclusive,
new Document.ActKey(toKey, Node.MAX_NODE, Integer.MAX_VALUE),
toInclusive
).values();
}
public Collection getActivationByRangeEnd(Range fromKey, boolean fromInclusive, Range toKey, boolean toInclusive) {
return activationsByRangeEnd.subMap(
new Document.ActKey(fromKey, Node.MIN_NODE, Integer.MIN_VALUE),
fromInclusive,
new Document.ActKey(toKey, Node.MAX_NODE, Integer.MAX_VALUE),
toInclusive
).values();
}
public Activation getNextActivation(Activation currentAct) {
Map.Entry me = currentAct == null ?
activationsById.firstEntry() :
activationsById.higherEntry(currentAct.id);
return me != null ? me.getValue() : null;
}
public int getNumberOfActivations() {
return activationsById.size();
}
@Override
public int compareTo(Document doc) {
return Integer.compare(id, doc.id);
}
public void propagate() {
boolean flag = true;
while(flag) {
queue.processChanges();
flag = ubQueue.process();
}
}
public void generateCandidates() {
TreeSet tmp = new TreeSet<>();
int i = 0;
if(!INCREMENTAL_MODE) {
candidates.clear();
}
for(Activation act: activationsById.subMap(INCREMENTAL_MODE ? lastProcessedActivationId : -1, false, Integer.MAX_VALUE, true).values()) {
if (act.decision == UNKNOWN && act.upperBound > 0.0) {
SearchNode.invalidateCachedDecision(act);
tmp.add(new Candidate(act, i++));
lastProcessedActivationId = Math.max(lastProcessedActivationId, act.id);
}
}
long v = visitedCounter++;
for(Activation act: inputNeuronActivations) {
act.markedHasCandidate = v;
}
while (!tmp.isEmpty()) {
int oldSize = tmp.size();
for (Candidate c : tmp) {
if (c.checkDependenciesSatisfied(v)) {
tmp.remove(c);
c.id = candidates.size();
candidates.add(c);
c.activation.markedHasCandidate = v;
break;
}
}
if(tmp.size() == oldSize) {
log.error("Cycle detected in the activations that is not marked recurrent.");
throw new RuntimeException("Cycle detected in the activations that is not marked recurrent.");
}
}
}
/**
* The method process needs to be called after all the input activations have been added to the
* network. It performs the search for the best interpretation.
*/
public void process() {
process(null);
}
public void process(Long timeoutInMilliSeconds) throws SearchNode.TimeoutException {
linker.lateLinking();
inputNeuronActivations.forEach(act -> vQueue.propagateActivationValue(0, act));
generateCandidates();
if(selectedSearchNode == null || !INCREMENTAL_MODE) {
selectedSearchNode = new SearchNode(this, null, null, 0);
}
SearchNode.search(this, selectedSearchNode, visitedCounter++, timeoutInMilliSeconds);
for(Activation act: activationsById.values()) {
if(act.isFinalActivation()) {
finallyActivatedNeurons.add(act.getINeuron());
}
}
if(SearchNode.COMPUTE_SOFT_MAX) {
computeSoftMax();
}
}
private void computeSoftMax() {
double norm = 0.0;
for(Double w: searchNodeWeights.values()) {
norm += Math.exp(w);
}
for(Activation act: activationsById.values()) {
if(act.searchStates != null) {
double avgValue = 0.0;
double avgPosValue = 0.0;
double avgP = 0.0;
double avgNet = 0.0;
double avgPosNet = 0.0;
for (Map.Entry me : searchNodeWeights.entrySet()) {
double p = Math.exp(me.getValue()) / norm;
Activation.State s = act.searchStates.get(me.getKey());
avgValue += p * s.value;
avgPosValue += p * s.posValue;
avgP += p * s.p;
avgNet += p * s.net;
avgPosNet += p * s.posNet;
}
act.avgState = new Activation.State(avgValue, avgPosValue, avgP, avgNet, avgPosNet, 0, 0.0);
}
}
}
public void dumpDebugCandidateStatistics() {
for (Candidate c : candidates) {
log.info(c.toString());
}
}
public void notifyWeightModified(Synapse synapse) {
Set is = modifiedWeights.get(synapse.output.get());
if(is == null) {
is = new TreeSet<>(Synapse.INPUT_SYNAPSE_COMP);
modifiedWeights.put(synapse.output.get(), is);
}
is.add(synapse);
}
/**
* Updates the model after the training step.
* It applies the weight and bias delta values and reflects the changes in the logic node structure.
*/
public void commit() {
modifiedWeights.forEach((n, inputSyns) -> Converter.convert(threadId, this, n, inputSyns));
modifiedWeights.clear();
}
/**
* Removes the activations of this document from the model again.
*/
public void clearActivations() {
activatedNeurons.forEach(n -> n.clearActivations(this));
activatedNodes.forEach(n -> n.clearActivations(this));
activationsById.clear();
addedNodeActivations.clear();
activatedNeurons.clear();
activatedNodes.clear();
addedNodes.clear();
if(model.lastCleanup[threadId] + CLEANUP_INTERVAL < id) {
model.lastCleanup[threadId] = id;
List> tmp;
synchronized(model.activeProviders) {
tmp = new ArrayList<>(model.activeProviders.values());
}
tmp.forEach(np -> {
AbstractNode an = np.getIfNotSuspended();
if (an != null && an instanceof Node) {
Node n = (Node) an;
Node.ThreadState th = n.threads[threadId];
if (th != null && th.lastUsed + CLEANUP_INTERVAL < id) {
n.threads[threadId] = null;
}
}
});
}
model.docs[threadId] = null;
}
public String generateOutputText() {
int oldLength = length();
TreeSet queue = new TreeSet<>(Comparator.comparingInt(p -> p.id));
for(Activation act: activationsByRangeBegin.values()) {
if(act.range.begin.getFinalPosition() != null && act.range.end.getFinalPosition() == null) {
queue.add(act.range.begin);
}
}
while(!queue.isEmpty()) {
Position pos = queue.pollFirst();
for(Activation act: pos.beginActivations) {
if (act.getINeuron().outputText != null && act.isFinalActivation()) {
String outText = act.getINeuron().outputText;
Position nextPos = act.range.end;
nextPos.setFinalPosition(pos.getFinalPosition() + outText.length());
content.replace(act.range.begin.getFinalPosition(), act.range.end.getFinalPosition(), outText);
queue.add(nextPos);
}
}
}
return content.substring(oldLength, length());
}
public String activationsToString() {
return activationsToString(true, false, false);
}
public String activationsToString(boolean finalOnly, boolean withTextSnippet, boolean withLogic) {
Set acts = new TreeSet<>(ACTIVATIONS_OUTPUT_COMPARATOR);
acts.addAll(activationsById.values());
StringBuilder sb = new StringBuilder();
sb.append("Activation ID -");
if(finalOnly) {
sb.append(" Final Decision -");
} else {
sb.append(" Decision -");
}
sb.append(" Range" + (withTextSnippet ? " | Text Snippet" : ""));
sb.append(" | Identity -");
sb.append(" Neuron Label -");
sb.append((withLogic ? " Logic Layer -" : ""));
sb.append(" Relational ID (Word Pos.) -");
sb.append(" Upper Bound -");
if(finalOnly) {
sb.append(" Final Value | Final Weight | Final Norm -");
} else {
sb.append(" Simulation Rounds [Round | Value | Weight | Norm] -");
}
sb.append(" Input Value |");
sb.append(" Target Value");
sb.append("\n");
sb.append("\n");
for(Activation act: acts) {
if(act.upperBound <= 0.0 && (act.targetValue == null || act.targetValue <= 0.0)) {
continue;
}
sb.append(act.toString(finalOnly, withTextSnippet, withLogic));
sb.append("\n");
}
if(selectedSearchNode != null) {
sb.append("\n Final SearchNode:" + selectedSearchNode.id + " WeightSum:" + selectedSearchNode.accumulatedWeight + "\n");
}
return sb.toString();
}
public Stream getAllActivationsStream() {
return addedNodeActivations.stream();
}
public class Queue {
public final TreeSet queue = new TreeSet<>(new Comparator() {
@Override
public int compare(Node n1, Node n2) {
int r = Integer.compare(n1.level, n2.level);
if(r != 0) return r;
ThreadState th1 = n1.getThreadState(threadId, true);
ThreadState th2 = n2.getThreadState(threadId, true);
return Long.compare(th1.queueId, th2.queueId);
}
});
private long queueIdCounter = 0;
public void add(Node n) {
ThreadState th = n.getThreadState(threadId, true);
if(!th.isQueued) {
th.isQueued = true;
th.queueId = queueIdCounter++;
queue.add(n);
}
}
public void processChanges() {
while(!queue.isEmpty()) {
Node n = queue.pollFirst();
ThreadState th = n.getThreadState(threadId, true);
th.isQueued = false;
n.processChanges(Document.this);
}
}
}
public class UpperBoundQueue {
public final ArrayDeque queue = new ArrayDeque<>();
public void add(Link l) {
if(!l.synapse.isRecurrent) {
add(l.output);
}
}
public void add(Activation act) {
if(!act.ubQueued && act.inputValue == null) {
act.ubQueued = true;
queue.addLast(act);
}
}
public boolean process() {
boolean flag = false;
while(!queue.isEmpty()) {
flag = true;
Activation act = queue.pollFirst();
act.ubQueued = false;
act.processBounds();
}
return flag;
}
}
private static Comparator VALUE_QUEUE_COMP = (a, b) -> {
int r = Integer.compare(a.getSequence(), b.getSequence());
if(r != 0) return r;
return Integer.compare(a.id, b.id);
};
public class ValueQueue {
public final ArrayList> queue = new ArrayList<>();
public void propagateActivationValue(int round, Activation act) {
act.getOutputLinks(false)
.forEach(l -> add(l.synapse.isRecurrent ? round + 1 : round, l.output));
}
private void add(Activation act) {
add(0, act);
act.getOutputLinks(false)
.filter(l -> l.synapse.isRecurrent)
.forEach(l -> add(0, l.output));
}
public void add(int round, Activation act) {
if(act.rounds.isQueued(round) || act.decision == Decision.UNKNOWN) return;
TreeSet q;
if(round < queue.size()) {
q = queue.get(round);
} else {
assert round == queue.size();
q = new TreeSet<>(VALUE_QUEUE_COMP);
queue.add(q);
}
act.rounds.setQueued(round, true);
q.add(act);
}
public double process(SearchNode sn) {
long v = visitedCounter++;
if(sn.getParent() != null && sn.getParent().candidate != null) {
add(sn.getParent().candidate.activation);
}
double delta = 0.0;
for(int round = 0; round < queue.size(); round++) {
TreeSet q = queue.get(round);
while (!q.isEmpty()) {
Activation act = q.pollFirst();
act.rounds.setQueued(round, false);
delta += act.process(sn, round, v);
}
}
return delta;
}
}
public void dumpOscillatingActivations() {
activatedNeurons.stream()
.flatMap(n -> n.getActivations(this, false).stream())
.filter(act -> act.rounds.getLastRound() != null && act.rounds.getLastRound() > MAX_ROUND - 5)
.forEach(act -> {
log.error(act.id + " " + act.range + " " + act.decision + " " + act.rounds);
log.error(act.linksToString());
log.error("");
});
}
}
aika-java-0.22/src/main/java/network/aika/Model.java 0000664 0000000 0000000 00000020674 13366231667 0022244 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import network.aika.lattice.InputNode;
import network.aika.lattice.Node;
import network.aika.neuron.INeuron;
import network.aika.neuron.Neuron;
import network.aika.Provider.SuspensionMode;
import network.aika.neuron.activation.Linker;
import network.aika.neuron.activation.SearchNode;
import java.lang.ref.WeakReference;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
/**
* The model consists of two layers. The first layer is the actual neural network consisting of neurons and synapses.
* The second layer is a pattern lattice containing a boolean logic representation of all the neurons. Whenever the
* synapse weights of a neuron are adjusted, then the underlying boolean logic representation of this neuron will be
* updated too.
*
*
The model supports parallel processing using a fixed number of threads.
*
* @author Lukas Molzberger
*/
public class Model {
public int numberOfThreads = 1;
public int[] lastCleanup;
public Document[] docs;
public SuspensionHook suspensionHook;
private WritableFactory nodeExtensionFactory;
private WritableFactory neuronExtensionFactory;
private WritableFactory synapseExtensionFactory;
private WritableFactory activationExtensionFactory;
public LinkerFactory linkerFactory = (doc) -> new Linker(doc);
public SearchNode.SkipSelectStep skipSelectStep = (act) -> false;
public AtomicInteger docIdCounter = new AtomicInteger(0);
public AtomicInteger currentId = new AtomicInteger(0);
// Important: the id field needs to be referenced by the provider!
public WeakHashMap>> providers = new WeakHashMap<>();
public Map> activeProviders = new TreeMap<>();
public Map passiveActivationFunctions = new TreeMap<>();
public int defaultThreadId = 0;
public static AtomicLong visitedCounter = new AtomicLong(1);
/**
* Creates a model with a single thread.
*/
public Model() {
this(null, 1);
}
public Model(SuspensionHook sh, int numberOfThreads) {
assert numberOfThreads >= 1;
this.numberOfThreads = numberOfThreads;
lastCleanup = new int[numberOfThreads];
docs = new Document[numberOfThreads];
suspensionHook = sh;
}
public SuspensionHook getSuspensionHook() {
return suspensionHook;
}
public void setSuspensionHook(SuspensionHook suspensionHook) {
this.suspensionHook = suspensionHook;
}
public WritableFactory getNodeExtensionFactory() {
return nodeExtensionFactory;
}
public void setNodeExtensionFactory(WritableFactory nodeExtensionFactory) {
this.nodeExtensionFactory = nodeExtensionFactory;
}
public WritableFactory getNeuronExtensionFactory() {
return neuronExtensionFactory;
}
public void setNeuronExtensionFactory(WritableFactory neuronExtensionFactory) {
this.neuronExtensionFactory = neuronExtensionFactory;
}
public WritableFactory getSynapseExtensionFactory() {
return synapseExtensionFactory;
}
public void setSynapseExtensionFactory(WritableFactory synapseExtensionFactory) {
this.synapseExtensionFactory = synapseExtensionFactory;
}
public WritableFactory getActivationExtensionFactory() {
return activationExtensionFactory;
}
public void setActivationExtensionFactory(WritableFactory activationExtensionFactory) {
this.activationExtensionFactory = activationExtensionFactory;
}
public LinkerFactory getLinkerFactory() {
return linkerFactory;
}
public void setLinkerFactory(LinkerFactory linkerFactory) {
this.linkerFactory = linkerFactory;
}
public SearchNode.SkipSelectStep getSkipSelectStep() {
return skipSelectStep;
}
public void setSkipSelectStep(SearchNode.SkipSelectStep skipSelectStep) {
this.skipSelectStep = skipSelectStep;
}
public Neuron createNeuron() {
return createNeuron(null);
}
public Neuron createNeuron(String label) {
return createNeuron(label, null);
}
public Neuron createNeuron(String label, String outputText) {
INeuron n = new INeuron(this, label, outputText);
InputNode iNode = InputNode.add(this, n);
iNode.setModified();
return n.provider;
}
public Document createDocument(String txt) {
return createDocument(txt, 0);
}
public Document createDocument(String txt, int threadId) {
Document doc = new Document(docIdCounter.addAndGet(1), txt, this, threadId);
if (txt != null) {
if (docs[threadId] != null) {
throw new RuntimeException("Two documents are using the same thread. Call clearActivations() first, before processing the next document.");
}
docs[threadId] = doc;
}
return doc;
}
public
> P lookupNodeProvider(int id) {
synchronized (providers) {
WeakReference> wr = providers.get(id);
if(wr != null) {
P p = (P) wr.get();
if (p != null) {
return p;
}
}
return (P) new Provider(this, id);
}
}
public Neuron lookupNeuron(int id) {
synchronized (providers) {
WeakReference> wr = providers.get(id);
if(wr != null) {
Neuron n = (Neuron) wr.get();
if (n != null) {
return n;
}
}
return new Neuron(this, id);
}
}
public void register(Provider p) {
synchronized (activeProviders) {
activeProviders.put(p.id, p);
}
}
public void unregister(Provider p) {
synchronized (activeProviders) {
activeProviders.remove(p.id);
}
}
/**
* Suspend all neurons and logic nodes whose last used document id is lower/older than {@param docId}.
*
* @param docId
*/
public void suspendUnusedNodes(int docId, SuspensionMode sm) {
docId = Math.min(docId, getOldestDocIdInProcessing());
List tmp;
synchronized (activeProviders) {
tmp = new ArrayList<>(activeProviders.values());
}
for (Provider p: tmp) {
suspend(docId, p, sm);
}
}
public int getOldestDocIdInProcessing() {
int oldestDocId = Integer.MAX_VALUE;
for(Document doc: docs) {
if(doc != null) oldestDocId = Math.min(oldestDocId, doc.id);
}
return oldestDocId;
}
/**
* Suspend all neurons and logic nodes in memory.
*
*/
public void suspendAll(SuspensionMode sm) {
suspendUnusedNodes(Integer.MAX_VALUE, sm);
}
private boolean suspend(int docId, Provider p, SuspensionMode sm) {
AbstractNode an = p.getIfNotSuspended();
if (an != null && an.lastUsedDocumentId < docId) {
p.suspend(sm);
return true;
}
return false;
}
public void removeProvider(Provider p) {
synchronized (activeProviders) {
activeProviders.remove(p.id);
}
synchronized (providers) {
providers.remove(p.id);
}
}
public interface WritableFactory {
Writable createObject();
}
public interface LinkerFactory {
Linker createLinker(Document doc);
}
}
aika-java-0.22/src/main/java/network/aika/PassiveInputFunction.java 0000664 0000000 0000000 00000000340 13366231667 0025330 0 ustar 00root root 0000000 0000000 package network.aika;
import network.aika.neuron.Synapse;
import network.aika.neuron.activation.Activation;
public interface PassiveInputFunction {
double getActivationValue(Synapse s, Activation outputActivation);
}
aika-java-0.22/src/main/java/network/aika/PatternDiscovery.java 0000664 0000000 0000000 00000006336 13366231667 0024510 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import network.aika.Document;
import network.aika.lattice.AndNode;
import network.aika.lattice.NodeActivation;
import java.util.ArrayList;
/**
*
* @author Lukas Molzberger
*/
public class PatternDiscovery {
public interface CandidateCheck {
/**
* Check if node is an interesting pattern that might be considered for further processing.
*
* This property is required to be monotonic over the size of the pattern. In other words, if a pattern is
* interesting, then all its sub patterns also need to be interesting.
*
* @param act
* @return
*/
boolean check(NodeActivation act, NodeActivation secondAct);
}
public interface PatternCheck {
boolean check(AndNode andNode);
}
public interface Counter {
/**
* Updates the statistics of this node
*
* @param act
* @return
*/
void count(NodeActivation act);
}
public static class Config {
public CandidateCheck candidateCheck;
public PatternCheck patternCheck;
public Counter counter;
public Config setCandidateCheck(CandidateCheck candidateCheck) {
this.candidateCheck = candidateCheck;
return this;
}
public Config setPatternCheck(PatternCheck patternCheck) {
this.patternCheck = patternCheck;
return this;
}
/**
* The counter callback function should implement a customized counting function.
* The counting function should modify the custom meta object stored in the node.
* The NodeStatisticFactory is used to instantiate the custom meta object for a node.
*
* @param counter
* @return
*/
public Config setCounter(Counter counter) {
this.counter = counter;
return this;
}
}
public static void discover(Document doc, Config config) {
doc.createV = doc.visitedCounter++;
doc.getAllActivationsStream().forEach(act -> config.counter.count(act));
ArrayList activations = new ArrayList<>(doc.addedNodeActivations);
doc.addedNodeActivations.clear();
activations.forEach(act -> act.node.discover(act, config));
// doc.propagate();
doc.addedNodeActivations.forEach(act -> config.counter.count(act));
}
}
aika-java-0.22/src/main/java/network/aika/Provider.java 0000664 0000000 0000000 00000006703 13366231667 0022773 0 ustar 00root root 0000000 0000000 package network.aika;
import java.io.*;
import java.lang.ref.WeakReference;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class Provider implements Comparable> {
public Model model;
public Integer id;
private volatile T n;
public enum SuspensionMode {
SAVE,
DISCARD
}
public Provider(Model model, int id) {
this.model = model;
this.id = id;
if(model != null) {
synchronized (model.providers) {
model.providers.put(this.id, new WeakReference<>(this));
}
}
}
public Provider(Model model, T n) {
this.model = model;
this.n = n;
id = model.suspensionHook != null ? model.suspensionHook.getNewId() : model.currentId.addAndGet(1);
synchronized (model.providers) {
model.providers.put(id, new WeakReference<>(this));
if(n != null) {
model.register(this);
}
}
}
public boolean isSuspended() {
return n == null;
}
public T getIfNotSuspended() {
return n;
}
public synchronized T get() {
if (n == null) {
reactivate();
}
return n;
}
public synchronized T get(int lastUsedDocumentId) {
T n = get();
n.lastUsedDocumentId = Math.max(n.lastUsedDocumentId, lastUsedDocumentId);
return n;
}
/**
*
* @param doc The document is used to remember when this node has been used last.
* @return
*/
public T get(Document doc) {
return doc != null ? get(doc.id) : get();
}
public synchronized void suspend(SuspensionMode sm) {
if(n == null) return;
assert model.suspensionHook != null;
n.suspend();
model.unregister(this);
if(sm == SuspensionMode.SAVE) {
save();
}
n = null;
}
public void save() {
if (n.modified) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (
GZIPOutputStream gzipos = new GZIPOutputStream(baos);
DataOutputStream dos = new DataOutputStream(gzipos);) {
n.write(dos);
} catch (IOException e) {
throw new RuntimeException(e);
}
model.suspensionHook.store(id, baos.toByteArray());
}
n.modified = false;
}
private void reactivate() {
assert model.suspensionHook != null;
byte[] data = model.suspensionHook.retrieve(id);
ByteArrayInputStream bais = new ByteArrayInputStream(data);
try (
GZIPInputStream gzipis = new GZIPInputStream(bais);
DataInputStream dis = new DataInputStream(gzipis);) {
n = (T) AbstractNode.read(dis, this);
} catch (IOException e) {
throw new RuntimeException(e);
}
n.reactivate();
model.register(this);
}
@Override
public boolean equals(Object o) {
return id == ((Provider) o).id;
}
@Override
public int hashCode() {
return id;
}
public String toString() {
return "p(" + id + ":" + (n != null ? n.toString() : "SUSPENDED") + ")";
}
public int compareTo(Provider n) {
if (id < n.id) return -1;
else if (id > n.id) return 1;
else return 0;
}
}
aika-java-0.22/src/main/java/network/aika/ReadWriteLock.java 0000664 0000000 0000000 00000005412 13366231667 0023674 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
/**
*
* @author Lukas Molzberger
*/
public class ReadWriteLock {
private int readers = 0;
private int writers = 0;
private int writeRequests = 0;
private long writerThreadId = -1;
private int waitForReadLock = 0;
private int waitForWriteLock = 0;
private Object writeLock = new Object();
public void acquireWriteLock() {
try {
synchronized (this) {
writeRequests++;
while (readers > 0) {
wait();
}
}
long tid = Thread.currentThread().getId();
synchronized (writeLock) {
if(writerThreadId != tid) {
waitForWriteLock++;
while (writers > 0) {
writeLock.wait();
}
waitForWriteLock--;
writerThreadId = tid;
}
writers++;
}
} catch(InterruptedException e) {
e.printStackTrace();
}
}
public synchronized void acquireReadLock() {
try {
waitForReadLock++;
while (writeRequests > 0) {
wait();
}
waitForReadLock--;
readers++;
} catch(InterruptedException e) {
e.printStackTrace();
}
}
public void releaseWriteLock() {
synchronized (writeLock) {
writers--;
if(writers == 0) {
writerThreadId = -1;
if(waitForWriteLock > 0) {
writeLock.notify();
}
}
}
synchronized(this) {
writeRequests--;
if(writeRequests == 0 && waitForReadLock > 0) {
notifyAll();
}
}
}
public synchronized void releaseReadLock() {
readers--;
if(writeRequests > 0 && readers == 0) {
notifyAll();
}
}
} aika-java-0.22/src/main/java/network/aika/SuspensionHook.java 0000664 0000000 0000000 00000002474 13366231667 0024171 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
/**
*
* The suspension hook is used to suspend neurons and logic nodes to an external storage in order to reduce the memory footprint.
*
* !!! Important: When using the suspension hook, all references to a neuron or a logic node need to occur through a
* provider. Otherwise the reference might be outdated.
*
* @author Lukas Molzberger
*/
public interface SuspensionHook {
int getNewId();
void store(int id, byte[] data);
byte[] retrieve(int id);
Iterable getAllNodeIds();
}
aika-java-0.22/src/main/java/network/aika/Utils.java 0000664 0000000 0000000 00000005475 13366231667 0022306 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import java.util.Comparator;
/**
*
* @author Lukas Molzberger
*/
public class Utils {
public static boolean contains(T[] in, T x, Comparator c) {
for(T y: in) {
if(c.compare(x, y) == 0) return true;
}
return false;
}
public static boolean compareNullSafe(Integer a, Integer b) {
if(a == null || b == null) return true;
return a >= b;
}
public static int compareInteger(Integer a, Integer b) {
if(a == null && b != null) return -1;
if(a != null && b == null) return 1;
if(a != null && b != null) {
return Integer.compare(a, b);
} else return 0;
}
public static double round(double x) {
return Math.round(x * 1000.0) / 1000.0;
}
public static Integer nullSafeAdd(Integer a, boolean fa, Integer b, boolean fb) {
if(b == null) return fa ? a : null;
if(a == null) return fb ? b : null;
else return a + b;
}
public static Integer nullSafeSub(Integer a, boolean fa, Integer b, boolean fb) {
if(b == null) return fa ? a : null;
if(a == null) return fb ? b : null;
return a - b;
}
public static Integer nullSafeMin(Integer a, Integer b) {
if(a == null) return b;
if(b == null) return a;
return Math.min(a, b);
}
public static Integer nullSafeMax(Integer a, Integer b) {
if(a == null) return b;
if(b == null) return a;
return Math.max(a, b);
}
public static Double nullSafeMax(Double a, Double b) {
if(a == null) return b;
if(b == null) return a;
return Math.max(a, b);
}
public static String collapseText(String txt, int length) {
if (txt.length() <= 2 * length) {
return txt;
} else {
return txt.substring(0, length) + "..." + txt.substring(txt.length() - length);
}
}
public static double sigmoid(double x) {
return 1.0 / (1.0 + Math.pow(Math.E, (-x)));
}
}
aika-java-0.22/src/main/java/network/aika/Writable.java 0000664 0000000 0000000 00000003034 13366231667 0022744 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
public interface Writable {
/**
* Serialize the fields of this object to out.
*
* @param out DataOuput to serialize this object into.
* @throws IOException
*/
void write(DataOutput out) throws IOException;
/**
* Deserialize the fields of this object from in.
*
*
For efficiency, implementations should attempt to re-use storage in the
* existing object where possible.
*
* @param in DataInput to deseriablize this object from.
* @throws IOException
*/
void readFields(DataInput in, Model m) throws IOException;
} aika-java-0.22/src/main/java/network/aika/lattice/ 0000775 0000000 0000000 00000000000 13366231667 0021755 5 ustar 00root root 0000000 0000000 aika-java-0.22/src/main/java/network/aika/lattice/AndNode.java 0000664 0000000 0000000 00000055127 13366231667 0024142 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika.lattice;
import network.aika.Document;
import network.aika.Model;
import network.aika.Provider;
import network.aika.Writable;
import network.aika.neuron.relation.Relation;
import network.aika.neuron.activation.Activation;
import network.aika.PatternDiscovery;
import network.aika.lattice.InputNode.InputActivation;
import network.aika.lattice.AndNode.AndActivation;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.util.*;
/**
* The {@code InputNode} and the {@code AndNode} classes together form a pattern lattice, containing all
* possible substructures of any given conjunction. For example if we have the conjunction ABCD where A, B, C, D are
* the inputs, then the pattern lattice will contain the nodes ABCD, ABC, ABD, ACD, BCD, AB, AC, AD, BC, BD, CD,
* A, B, C, D. The pattern lattice is organized in layers, where each layer only contains conjunctions/patterns of the
* same size. These layers are connected through refinements. For example the and-node
* ABD on layer 3 is connected to the and-node ABCD on layer 4 via the refinement C.
*
* @author Lukas Molzberger
*/
public class AndNode extends Node {
public List parents;
public AndNode() {
parents = new ArrayList<>();
}
public AndNode(Model m, int level, List parents) {
super(m, level);
this.parents = parents;
}
private void init() {
for(Entry e: parents) {
e.rv.child = provider;
Node pn = e.rv.parent.get();
pn.addAndChild(e.ref, e.rv);
pn.setModified();
}
}
public void propagate(AndActivation act) {
apply(act);
}
@Override
public void cleanup() {
if(!isRemoved && !isRequired()) {
remove();
for(Entry e: parents) {
e.rv.parent.get().cleanup();
}
}
}
@Override
void processActivation(AndActivation act) {
if(act.isComplete()) {
super.processActivation(act);
}
}
@Override
void apply(AndActivation act) {
if (andChildren != null) {
for (Link fl : act.inputs) {
if(fl == null) continue;
InputActivation refAct = fl.refAct;
Refinement ref = fl.ref;
RefValue rv = fl.rv;
NodeActivation pAct = fl.input;
for (Link sl : pAct.outputsToAndNode.values()) {
InputActivation secondRefAct = sl.refAct;
Refinement secondRef = sl.ref;
RefValue secondRv = sl.rv;
NodeActivation secondAct = sl.output;
if (act != secondAct) {
Relation[] relations = new Relation[secondRef.relations.length() + 1];
for(int i = 0; i < secondRef.relations.length(); i++) {
relations[rv.offsets[i]] = secondRef.relations.get(i);
}
lock.acquireReadLock();
for(Map.Entry me: andChildren.subMap(
new Refinement(RelationsMap.MIN, secondRef.input),
new Refinement(RelationsMap.MAX, secondRef.input)).entrySet()) {
Refinement nRef = me.getKey();
RefValue nRv = me.getValue();
if(nRef.contains(secondRef, rv)) {
AndNode nlNode = nRv.child.get(act.doc);
AndActivation nlAct = lookupAndActivation(act, nRef);
if(nlAct == null) {
nlAct = new AndActivation(act.doc.logicNodeActivationIdCounter++, act.doc, nlNode);
nlAct.link(nRef, nRv, secondRefAct, act);
}
nlAct.node.addActivation(nlAct);
for(Entry secondNE: nlNode.parents) {
if(secondNE.rv.parent.get(act.doc) == secondAct.node && secondNE.ref.contains(ref, secondRv)) {
nlAct.link(secondNE.ref, secondNE.rv, refAct, secondAct);
break;
}
}
}
}
lock.releaseReadLock();
}
}
}
}
OrNode.processCandidate(this, act, false);
}
private AndActivation lookupAndActivation(NodeActivation input, Refinement ref) {
for (Link l : input.outputsToAndNode.values()) {
if(l.ref.compareTo(ref) == 0) {
return l.output;
}
}
return null;
}
@Override
public void discover(AndActivation act, PatternDiscovery.Config config) {
Document doc = act.doc;
for(Link fl : act.inputs) {
if(fl == null) continue;
for (Link sl : fl.input.outputsToAndNode.values()) {
AndActivation secondAct = sl.output;
if (secondAct.node instanceof AndNode) {
if (act != secondAct && config.candidateCheck.check(act, secondAct)) {
Activation iAct = act.getInputActivation(fl.rv.refOffset);
Activation secondIAct = secondAct.getInputActivation(sl.rv.refOffset);
List rels = InputNode.getRelations(iAct, secondIAct);
rels.add(null);
for(Relation rel: rels) {
Refinement nRef = createRefinement(fl.rv, sl.ref, rel);
AndNode.RefValue rv = extend(doc.threadId, doc, nRef, config);
if (rv != null) {
AndNode nln = rv.child.get();
nln.isDiscovered = true;
}
}
}
}
}
}
}
private Refinement createRefinement(RefValue firstRV, Refinement secondRef, Relation rel) {
Relation[] srm = secondRef.relations.relations;
RelationsMap rm = new RelationsMap();
rm.relations = new Relation[srm.length + 1];
for (int i = 0; i < srm.length; i++) {
rm.relations[firstRV.offsets[i]] = srm[i];
}
rm.relations[firstRV.refOffset] = rel;
return new Refinement(rm, secondRef.input);
}
public RefValue extend(int threadId, Document doc, Refinement firstRef, PatternDiscovery.Config patterDiscoverConfig) {
if(firstRef.relations.size() == 0) return null;
RefValue firstRV = getAndChild(firstRef);
if(firstRV != null) {
return firstRV;
}
int firstRefOffset = level;
Integer[] firstOffsets = new Integer[level];
for(int i = 0; i < firstOffsets.length; i++) {
firstOffsets[i] = i;
}
List nextLevelParents = new ArrayList<>();
for(Entry firstParent: parents) {
Node parentNode = firstParent.rv.parent.get(doc);
Relation[] secondParentRelations = new Relation[firstRef.relations.length() - 1];
for(int i = 0; i < firstRef.relations.length(); i++) {
Integer j = firstParent.rv.reverseOffsets[i];
if(j != null) {
secondParentRelations[j] = firstRef.relations.get(i);
}
}
Refinement secondParentRef = new Refinement(new RelationsMap(secondParentRelations), firstRef.input);
RefValue secondParentRV = patterDiscoverConfig != null ?
parentNode.getAndChild(secondParentRef) :
parentNode.extend(threadId, doc, secondParentRef, null);
if(secondParentRV == null) {
continue;
}
Relation[] secondRelations = new Relation[firstParent.ref.relations.length() + 1];
for(int i = 0; i < firstParent.ref.relations.length(); i++) {
int j = secondParentRV.offsets[i];
secondRelations[j] = firstParent.ref.relations.get(i);
}
Relation rel = firstRef.relations.get(firstParent.rv.refOffset);
if(rel != null) {
secondRelations[secondParentRV.refOffset] = rel.invert();
}
Refinement secondRef = new Refinement(new RelationsMap(secondRelations), firstParent.ref.input);
Integer[] secondOffsets = new Integer[secondParentRV.offsets.length + 1];
for(int i = 0; i < firstParent.rv.reverseOffsets.length; i++) {
Integer j = firstParent.rv.reverseOffsets[i];
if(j != null) {
secondOffsets[secondParentRV.offsets[j]] = i;
}
}
secondOffsets[secondParentRV.refOffset] = firstRefOffset;
nextLevelParents.add(new Entry(secondRef, new RefValue(secondOffsets, firstOffsets[firstParent.rv.refOffset], secondParentRV.child)));
}
firstRV = new RefValue(firstOffsets, firstRefOffset, provider);
nextLevelParents.add(new Entry(firstRef, firstRV));
return createAndNode(provider.model, doc, nextLevelParents, level + 1, patterDiscoverConfig) ? firstRV : null;
}
static boolean createAndNode(Model m, Document doc, List parents, int level, PatternDiscovery.Config patterDiscoverConfig) {
if (parents != null) {
// Locking needs to take place in a predefined order.
TreeSet> parentsForLocking = new TreeSet();
for(Entry e: parents) {
parentsForLocking.add(e.rv.parent);
}
for (Provider pn : parentsForLocking) {
pn.get().lock.acquireWriteLock();
}
try {
AndNode nln = new AndNode(m, level, parents);
if(patterDiscoverConfig != null && !patterDiscoverConfig.patternCheck.check(nln)) {
return false;
}
nln.init();
nln.postCreate(doc);
} finally {
for (Provider pn : parentsForLocking) {
pn.get().lock.releaseWriteLock();
}
}
}
return true;
}
@Override
public void changeNumberOfNeuronRefs(int threadId, long v, int d) {
super.changeNumberOfNeuronRefs(threadId, v, d);
parents.forEach(e -> e.rv.parent.get().changeNumberOfNeuronRefs(threadId, v, d));
}
@Override
public void reprocessInputs(Document doc) {
for(Entry e: parents) {
Node> pn = e.rv.parent.get();
for(NodeActivation act : pn.getActivations(doc)) {
act.repropagateV = markedCreated;
act.node.propagate(act);
}
}
}
@Override
public void remove() {
super.remove();
for(Entry e: parents) {
Node pn = e.rv.parent.get();
pn.lock.acquireWriteLock();
pn.removeAndChild(e.ref);
pn.setModified();
pn.lock.releaseWriteLock();
}
}
public String logicToString() {
StringBuilder sb = new StringBuilder();
sb.append("AND(" + level + ")[");
boolean first = true;
for(Entry e: parents) {
if(!first) {
sb.append(",");
}
first = false;
sb.append(e.ref);
}
sb.append("]");
return sb.toString();
}
@Override
public void write(DataOutput out) throws IOException {
out.writeBoolean(false);
out.writeChar('A');
super.write(out);
out.writeInt(parents.size());
for(Entry e: parents) {
e.ref.write(out);
e.rv.write(out);
}
}
@Override
public void readFields(DataInput in, Model m) throws IOException {
super.readFields(in, m);
int s = in.readInt();
for(int i = 0; i < s; i++) {
Refinement ref = Refinement.read(in, m);
RefValue rv = RefValue.read(in, m);
parents.add(new Entry(ref, rv));
}
}
/**
*
*/
public static class Refinement implements Comparable, Writable {
public RelationsMap relations;
public Provider input;
private Refinement() {}
public Refinement(RelationsMap relations, Provider input) {
this.relations = relations;
this.input = input;
}
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("(");
sb.append(relations);
sb.append(input.get().logicToString());
sb.append(")");
return sb.toString();
}
public void write(DataOutput out) throws IOException {
relations.write(out);
out.writeInt(input.id);
}
public void readFields(DataInput in, Model m) throws IOException {
relations = RelationsMap.read(in, m);
input = m.lookupNodeProvider(in.readInt());
}
public static Refinement read(DataInput in, Model m) throws IOException {
Refinement k = new Refinement();
k.readFields(in, m);
return k;
}
@Override
public int compareTo(Refinement ref) {
int r = input.compareTo(ref.input);
if(r != 0) return r;
return relations.compareTo(ref.relations);
}
public boolean contains(Refinement ref, RefValue rv) {
for(int i = 0; i < ref.relations.length(); i++) {
Relation ra = ref.relations.get(i);
Relation rb = relations.get(rv.offsets[i]);
if((ra == null && rb != null) || (ra != null && rb == null)) return false;
if(ra != null && rb != null && ra.compareTo(rb) != 0) {
return false;
}
}
return true;
}
}
public static class RelationsMap implements Comparable, Writable {
public static final RelationsMap MIN = new RelationsMap();
public static final RelationsMap MAX = new RelationsMap();
public Relation[] relations;
public RelationsMap() {}
public RelationsMap(Relation[] relations) {
this.relations = relations;
}
public void write(DataOutput out) throws IOException {
out.writeInt(relations.length);
for(int i = 0; i < relations.length; i++) {
Relation rel = relations[i];
out.writeBoolean(rel != null);
if(rel != null) {
rel.write(out);
}
}
}
public void readFields(DataInput in, Model m) throws IOException {
int l = in.readInt();
relations = new Relation[l];
for(int i = 0; i < l; i++) {
if(in.readBoolean()) {
relations[i] = Relation.read(in, m);
}
}
}
public static RelationsMap read(DataInput in, Model m) throws IOException {
RelationsMap k = new RelationsMap();
k.readFields(in, m);
return k;
}
public String toString() {
StringBuilder sb = new StringBuilder();
for(int i = 0; i < relations.length; i++) {
Relation rel = relations[i];
if(rel != null) {
sb.append(i + ":" + rel + ", ");
}
}
return sb.toString();
}
@Override
public int compareTo(RelationsMap rm) {
if (this == MIN) return -1;
if (rm == MIN) return 1;
if (this == MAX) return 1;
if (rm == MAX) return -1;
int r = Integer.compare(relations.length, rm.relations.length);
if(r != 0) return r;
for(int i = 0; i < relations.length; i++) {
Relation ra = relations[i];
Relation rb = rm.relations[i];
if(ra == null && rb == null) continue;
if(ra == null && rb != null) return -1;
if(ra != null && rb == null) return 1;
r = Relation.COMPARATOR.compare(ra, rb);
if(r != 0) return r;
}
return 0;
}
public int length() {
return relations.length;
}
public Relation get(int i) {
return relations[i];
}
public int size() {
if(relations.length == 0) return 0;
int count = 0;
for(int i = 0; i < relations.length; i++) {
if(relations[i] != null) count++;
}
return count;
}
public boolean isExact() {
for(Relation rel: relations) {
if(!rel.isExact()) {
return false;
}
}
return true;
}
}
public static class RefValue implements Writable {
public Integer[] offsets; // input offsets -> output offsets
public Integer[] reverseOffsets; // output offsets -> input offsets
public int refOffset;
public Provider parent;
public Provider child;
private RefValue() {}
public RefValue(Integer[] offsets, int refOffset, Provider parent) {
this.offsets = offsets;
reverseOffsets = new Integer[offsets.length + 1];
for(int i = 0; i < offsets.length; i++) {
reverseOffsets[offsets[i]] = i;
}
this.refOffset = refOffset;
this.parent = parent;
}
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(offsets.length);
for(int i = 0; i < offsets.length; i++) {
Integer ofs = offsets[i];
out.writeBoolean(ofs != null);
out.writeInt(ofs);
}
out.writeInt(refOffset);
out.writeInt(parent.id);
out.writeInt(child.id);
}
public static RefValue read(DataInput in, Model m) throws IOException {
RefValue rv = new RefValue();
rv.readFields(in, m);
return rv;
}
@Override
public void readFields(DataInput in, Model m) throws IOException {
int l = in.readInt();
offsets = new Integer[l];
reverseOffsets = new Integer[l + 1];
for(int i = 0; i < l; i++) {
if(in.readBoolean()) {
Integer ofs = in.readInt();
offsets[i] = ofs;
reverseOffsets[ofs] = i;
}
}
refOffset = in.readInt();
parent = m.lookupNodeProvider(in.readInt());
child = m.lookupNodeProvider(in.readInt());
}
}
public static class Entry {
public Refinement ref;
public RefValue rv;
public Entry(Refinement ref, RefValue rv) {
this.ref = ref;
this.rv = rv;
}
}
public static class AndActivation extends NodeActivation {
public Link[] inputs;
public AndActivation(int id, Document doc, AndNode node) {
super(id, doc, node);
inputs = new Link[node.level];
}
public void link(Refinement ref, RefValue rv, InputActivation refAct, NodeActivation input) {
Link l = new Link(ref, rv, refAct, input, this);
inputs[rv.refOffset] = l;
input.outputsToAndNode.put(id, l);
}
public Activation getInputActivation(int i) {
Link l = inputs[i];
if(l != null) {
return l.refAct.input.input;
} else {
for(int j = 0; j < inputs.length; j++) {
if (j != i) {
l = inputs[j];
if(l != null) {
return l.input.getInputActivation(l.rv.reverseOffsets[i]);
}
}
}
return null;
}
}
public boolean isComplete() {
int numberOfLinks = 0;
for (Link l : inputs) {
if (l != null) numberOfLinks++;
}
return node.parents.size() == numberOfLinks;
}
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("A-ACT(");
boolean first = true;
for(int i = 0; i < inputs.length; i++) {
Activation iAct = getInputActivation(i);
if(iAct != null) {
if(!first) {
sb.append(",");
}
sb.append(i + ":" + iAct.getLabel() + " " + iAct.range + " (" + iAct.id + ")");
first = false;
}
}
sb.append(")");
return sb.toString();
}
}
public static class Link {
public Refinement ref;
public RefValue rv;
public NodeActivation input;
public InputActivation refAct;
public AndActivation output;
public Link(Refinement ref, RefValue rv, InputActivation refAct, NodeActivation input, AndActivation output) {
this.ref = ref;
this.rv = rv;
this.refAct = refAct;
this.input = input;
this.output = output;
}
}
}
aika-java-0.22/src/main/java/network/aika/lattice/InputNode.java 0000664 0000000 0000000 00000027540 13366231667 0024535 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika.lattice;
import network.aika.Document;
import network.aika.Model;
import network.aika.Provider;
import network.aika.neuron.INeuron;
import network.aika.neuron.Neuron;
import network.aika.neuron.range.Range;
import network.aika.neuron.relation.AncestorRelation;
import network.aika.neuron.relation.RangeRelation;
import network.aika.neuron.relation.Relation;
import network.aika.neuron.activation.Activation;
import network.aika.PatternDiscovery;
import network.aika.lattice.AndNode.AndActivation;
import network.aika.lattice.InputNode.InputActivation;
import network.aika.lattice.AndNode.Refinement;
import network.aika.lattice.AndNode.RefValue;
import network.aika.lattice.AndNode.RelationsMap;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.util.*;
import static network.aika.neuron.range.Range.Relation.*;
/**
* The {@code InputNode} class is the input layer for the boolean logic. The input-node has two sources of
* activations. First, it might be underlying logic node of an {@code InputNeuron} in which case the input
* activations come from the outside. The second option is that the activation come from the output of another neuron.
*
* @author Lukas Molzberger
*/
public class InputNode extends Node {
public Neuron inputNeuron;
public TreeMap nonExactAndChildren;
private long visitedDiscover;
public static final Relation[] CANDIDATE_RELATIONS = new Relation[] {
new RangeRelation(EQUALS),
new RangeRelation(BEGIN_TO_END_EQUALS),
new RangeRelation(END_TO_BEGIN_EQUALS),
new RangeRelation(BEGIN_EQUALS),
new RangeRelation(END_EQUALS),
new RangeRelation(CONTAINS),
new RangeRelation(CONTAINED_IN),
new AncestorRelation(AncestorRelation.Type.IS_DESCENDANT_OF),
new AncestorRelation(AncestorRelation.Type.IS_ANCESTOR_OF),
new AncestorRelation(AncestorRelation.Type.COMMON_ANCESTOR)
};
public InputNode() {
}
public InputNode(Model m) {
super(m, 1);
}
public static InputNode add(Model m, INeuron input) {
if (input.outputNode != null) {
return input.outputNode.get();
}
InputNode in = new InputNode(m);
if (input != null && in.inputNeuron == null) {
in.inputNeuron = input.provider;
input.outputNode = in.provider;
input.setModified();
}
return in;
}
public void addActivation(Activation inputAct) {
if(inputAct.repropagateV != null && inputAct.repropagateV != markedCreated) return;
InputActivation act = new InputActivation(inputAct.doc.logicNodeActivationIdCounter++, inputAct, this);
addActivation(act);
}
public void propagate(InputActivation act) {
apply(act);
}
public void reprocessInputs(Document doc) {
inputNeuron.get(doc).getActivations(doc, false).forEach(act -> {
act.repropagateV = markedCreated;
if(act.upperBound > 0.0) {
act.getINeuron().propagate(act);
}
});
}
void addAndChild(AndNode.Refinement ref, AndNode.RefValue child) {
super.addAndChild(ref, child);
if(!ref.relations.isExact()) {
if (nonExactAndChildren == null) {
nonExactAndChildren = new TreeMap<>();
}
AndNode.RefValue n = nonExactAndChildren.put(ref, child);
assert n == null;
}
}
void removeAndChild(AndNode.Refinement ref) {
super.removeAndChild(ref);
if(!ref.relations.isExact()) {
if (nonExactAndChildren != null) {
nonExactAndChildren.remove(ref);
if (nonExactAndChildren.isEmpty()) {
nonExactAndChildren = null;
}
}
}
}
public RefValue extend(int threadId, Document doc, Refinement ref, PatternDiscovery.Config patterDiscoveryConfig) {
if(ref.relations.size() == 0) return null;
Relation rel = ref.relations.get(0);
if(rel == null) {
return null;
}
RefValue rv = getAndChild(ref);
if(rv != null) {
return rv;
}
List nlParents = new ArrayList<>();
Refinement mirrorRef = new Refinement(new AndNode.RelationsMap(new Relation[]{rel.invert()}), provider);
nlParents.add(new AndNode.Entry(mirrorRef, new RefValue(new Integer[] {1}, 0, ref.input)));
rv = new RefValue(new Integer[] {0}, 1, provider);
nlParents.add(new AndNode.Entry(ref, rv));
return AndNode.createAndNode(provider.model, doc, nlParents, level + 1, patterDiscoveryConfig) ? rv : null;
}
/**
* @param act
*/
@Override
void apply(InputActivation act) {
try {
lock.acquireReadLock();
if (andChildren != null) {
TreeMap children;
if(andChildren.size() > 10) {
children = nonExactAndChildren;
applyExactRelations(act);
} else {
children = andChildren;
}
if(children != null) {
children.forEach((ref, rv) -> {
InputNode in = ref.input.getIfNotSuspended();
if (in != null) {
addNextLevelActivations(in, ref, rv.child.get(act.doc), act);
}
});
}
}
} finally {
lock.releaseReadLock();
}
OrNode.processCandidate(this, act, false);
}
private void applyExactRelations(InputActivation act) {
Activation iAct = act.input.input;
for(Range.Relation rel: new Range.Relation[] {BEGIN_EQUALS, END_EQUALS, BEGIN_TO_END_EQUALS, END_TO_BEGIN_EQUALS}) {
for(Activation linkedAct: RangeRelation.getActivationsByRangeEquals(act.doc, iAct.range, rel)) {
Provider in = linkedAct.getINeuron().outputNode;
for (Map.Entry me : andChildren.subMap(
new Refinement(RelationsMap.MIN, in),
new Refinement(RelationsMap.MAX, in)).entrySet()) {
addNextLevelActivations(in.get(act.doc), me.getKey(), me.getValue().child.get(act.doc), act);
}
}
}
}
private static void addNextLevelActivations(InputNode secondNode, Refinement ref, AndNode nln, InputActivation act) {
Document doc = act.doc;
INeuron.ThreadState th = secondNode.inputNeuron.get().getThreadState(doc.threadId, false);
if (th == null || th.isEmpty()) return;
Activation iAct = act.input.input;
if(act.repropagateV != null && act.repropagateV != nln.markedCreated) return;
ref.relations.get(0).getActivations(secondNode.inputNeuron.get(doc), iAct).forEach(secondIAct -> {
if (secondIAct.outputToInputNode != null) {
InputActivation secondAct = secondIAct.outputToInputNode.output;
if (secondAct != null) {
// if (!Conflicts.isConflicting(iAct, secondIAct)) {
AndActivation oAct = new AndActivation(doc.logicNodeActivationIdCounter++, doc, nln);
for (AndNode.Entry e : nln.parents) {
boolean match = e.ref.compareTo(ref) == 0;
oAct.link(e.ref, e.rv, match ? secondAct : act, match ? act : secondAct);
}
nln.addActivation(oAct);
// }
}
}
}
);
}
@Override
public void discover(InputActivation act, PatternDiscovery.Config config) {
if(!act.input.input.isFinalActivation()) {
return;
}
Document doc = act.doc;
doc.getActivations(true).forEach(secondNAct -> {
InputActivation secondAct = secondNAct.outputToInputNode.output;
if (act != secondAct && config.candidateCheck.check(act, secondAct)) {
List relations = getRelations(act.input.input, secondNAct);
for(Relation r: relations) {
InputNode in = secondAct.node;
if (r != null) {
RelationsMap rm = new RelationsMap(new Relation[] {r});
Refinement ref = new Refinement(rm, in.provider);
AndNode.RefValue rv = extend(doc.threadId, doc, ref, config);
if (rv != null) {
AndNode nln = rv.child.get();
nln.isDiscovered = true;
}
}
}
}
});
}
public static List getRelations(Activation act1, Activation act2) {
ArrayList rels = new ArrayList<>();
for(Relation rel: CANDIDATE_RELATIONS) {
if(rel.test(act2, act1)) {
rels.add(rel);
break;
}
}
return rels;
}
@Override
public void cleanup() {
}
public String logicToString() {
StringBuilder sb = new StringBuilder();
sb.append("I");
sb.append("[");
if (inputNeuron != null) {
sb.append(inputNeuron.id);
if (inputNeuron.getLabel() != null) {
sb.append(",");
sb.append(inputNeuron.getLabel());
}
}
sb.append("]");
return sb.toString();
}
@Override
public void write(DataOutput out) throws IOException {
out.writeBoolean(false);
out.writeChar('I');
super.write(out);
out.writeBoolean(inputNeuron != null);
if (inputNeuron != null) {
out.writeInt(inputNeuron.id);
}
}
@Override
public void readFields(DataInput in, Model m) throws IOException {
super.readFields(in, m);
if (in.readBoolean()) {
inputNeuron = m.lookupNeuron(in.readInt());
}
}
public static class InputActivation extends NodeActivation {
public Link input;
public InputActivation(int id, Activation iAct, InputNode node) {
super(id, iAct.doc, node);
input = new Link(iAct, this);
iAct.outputToInputNode = input;
}
public Activation getInputActivation(int i) {
assert i == 0;
return input.input;
}
public String toString() {
return "I-ACT(" + input.input.getLabel() + " " + input.input.range + ")";
}
}
public static class Link {
public Activation input;
public InputActivation output;
public Link(Activation iAct, InputActivation oAct) {
input = iAct;
output = oAct;
}
}
}
aika-java-0.22/src/main/java/network/aika/lattice/Node.java 0000664 0000000 0000000 00000027030 13366231667 0023507 0 ustar 00root root 0000000 0000000 /*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package network.aika.lattice;
import network.aika.*;
import network.aika.PatternDiscovery;
import network.aika.Document;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
/**
* The {@code Node} class is the abstract class for all the boolean logic nodes underneath the neural network layer.
* These nodes form a boolean representation for all the neurons of the neural network. Whenever changes occur to the
* synapse weights in the neural layer, then the structure of the boolean representation needs to be adjusted. Several
* neurons, however, might share common substructures in this boolean representation. The {@code InputNode} and
* the {@code AndNode} classes together form a pattern lattice, containing all possible substructures of any
* given conjunction. For example if we have the conjunction ABCD where A, B, C, D are the inputs then the
* pattern lattice will contain the nodes ABCD, ABC, ABD, ACD, BCD, AB, AC, AD, BC, BD, CD, A, B, C, D. The class
* {@code OrNode} is a disjunction of either input-nodes or and-nodes. The or-node is connected with one of
* the neurons.
*