HTTP-APIs · gustavodemorais · Feb 5, 2020 · Jul 8, 2019 · Jul 19, 2019 · Aug 22, 2019
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -6,7 +6,7 @@
 
 * Read this [how-to about writing a PR](https://github.com/blog/1943-how-to-write-the-perfect-pull-request) and this [other how-to about writing a issue](https://wiredcraft.com/blog/how-we-write-our-github-issues/)
 
-* **first ask in chat**: if you find a problem, first ask for [help in the chat](https://gitter.im/HTTP-APIs/Lobby), then consider opening a issue.
+* **first ask in chat**: if you find a problem, first ask for [help in the chat](https://hydraecosystem.slack.com/) and then go through the previous issues. If the problem still persists, only then consider opening an issue.
 
 * **read history**: before opening a PR be sure that all the tests pass successfully. If any is failing for non-related reasons, annotate the test failure in the PR comment.
 

diff --git a/.gitignore b/.gitignore
@@ -112,7 +112,7 @@ Pipfile.lock
 .mypy_cache/
 
 # build packages
-src 
+/src 
 
 # Graphviz generated graph
 hydra_graph.gv
diff --git a/.travis.yml b/.travis.yml
@@ -3,7 +3,7 @@ services:
   - docker
 
 before_script:
-  - docker run -d -p 6379:6379 -it --rm --name redisgraph redislabs/redisgraph:edge
+  - docker run -d -p 6379:6379 -it --rm --name redisgraph redislabs/redisgraph:2.0-edge
 
 python:
   - "3.5"

diff --git a/LICENSE b/LICENSE
@@ -1,6 +1,7 @@
 MIT License
 
 Copyright (c) 2018 HYDRA W3C Group, Github.com/HTTP-APIs contributors
+Copyright (c) 2019 Github.com/HTTP-APIs contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/README.md b/README.md
@@ -4,254 +4,131 @@ For a general introduction to Hydra Ecosystem, see [hydraecosystem.org](http://h
 
 `hydra-python-agent` is a smart Hydra client implemented in Python which works with [hydrus](https://github.com/HTTP-APIs/hydrus). Reference implementation is [Heracles.ts](https://github.com/HydraCG/Heracles.ts). Smart clients are generic automated clients that establish resilient connected data networks leveraging knowledge graphs.
 
+## Quick Start
+Our Hydra Agent has different interfaces that you can try:
+
+- [A Web-based GUI](https://github.com/HTTP-APIs/hydra-python-agent-gui/tree/agent-gui-1.0/console-frontend) - which shows how Hydra APIs are connected and how the Smart Agent is generic and can automatically build requests.  
+- [A Python package](https://github.com/HTTP-APIs/hydra-python-agent/#user-content-agent-package) - so you can use to communicate with a Hydra API in your code/software. 
+- [Natural-language-like command line tool](#natural-language-like-command-line-tool) - this is still a GET only implementation
+
 ## General characteristics
 
-The client is designed to:
-* Cache metadata from the Hydra server it connects to, to allow querying on the client-side;
-* Use Redis as a graph-store leveraging `redisgraph` (see [here](https://oss.redislabs.com/redisgraph/));
-* simply, metadata and data are loaded from the server and stored in Redis;
+The Agent is *designed* to:
+* Provide a seamless Client that can be used to communicate with Hydra APIs
+* Cache metadata from the Hydra server it connects to, to allow querying on the client-side
+* Maintain a syncrhonization mechanism which makes sure cached resources are consistent
 * The graph can be queried using OpenCypher.
 
-The starting objective is to create a querying layer that is able to reach data in one or more Hydra srever/s. Leveraging Redis, clients can construct their own representation of the data stored in one or more Hydra servers; querying the data as they need it, and respond complex semantic queries. This will allow any client connected to any server to have access to an "aggregated view" of the connected network (the network of all the servers it connects to). 
-
-## Missing bits at the moment
-* For now it is a proof-of-concept, only `GET` functionality
-* Soon to develop, a reliable synchronization mechanism to allow strong consistency between server-side data and client-side representation ([#98](https://github.com/HTTP-APIs/hydra-python-agent/issues/98)).
-* Allow users to interact with the server using Natural Language which is a processed machine consumable language. **(under development)**
+The final goal is to create a Client that can connected to multiple hydrus servers and operate between them while caching information in a graph-based database(Redis). This should enable the client to have an "aggregated view" of the connected network (the network of all the servers it connects to) and make complex sematic queries to it.
 
 ## Installation
 
-**NOTE:** You'll need to use python3.
+**NOTE:** You'll need to use python3. Using venv(virtual environment) is recommended.
 
-To install only requirements:
+Clone and setup a virtual environment:
 
-    pip3 install -r requirements.txt
-
-or,
+    git clone https://github.com/HTTP-APIs/hydra-python-agent.git
+    cd hydra-python-agent
+    python3 -m venv venv
+    source venv/bin/activate
 
-To install or setup the client environment, you have to run:
-
-     python3 setup.py install
+Install dependencies and setup Agent:
 
+    pip3 install --upgrade pip
+    pip3 install -r requirements.txt
+    python3 setup.py install
 
-To install Redis and other Redis modules:
+Setup Redis which is used as caching layer(if permission denied use `sudo`):
 
     ./redis_setup.sh
 
-## Quickstart
-
-### Demo
-
-To run the demo for hydra-python-agent, you have to follow the instructions:
-
-* Clone the repo:
-
-        git clone https://github.com/HTTP-APIs/hydra-python-agent.git
-
-* Change directory and switch to the develop branch:
-
-        cd hydra-python-agent
-        git checkout -b develop origin/develop
-
-* Now to install the requirements or setup the environment:
-
-    you should follow the instructions of [installation](#installation).
-
-After setup the environment. You can query or run the client.
-
-* To run both the things Redis server and the client. You can run the command:
-
-        docker-compose run client
-
-
-    and provide a valid URL(of a hydrus updated server) and then you can query in querying format.
-
-        `>>>url` #here url should be a valid link, for testing you can use http://35.224.198.158:8080/api
-        `>>>help` # it will provide the querying format
-
-**Obs.: If failing to connect to localhost** running the Agent via Docker, head to [issue #104](https://github.com/HTTP-APIs/hydra-python-agent/issues/104#issuecomment-497381440).
-
-#### Code simplification
-
-To create the graph in Redis memory, use(graph_init.py) :
-```
-    import redis
-    from redisgraph import Graph, Node, Edge
-    redis_con = redis.Redis(host='localhost', port=6379)
-    self.redis_graph = Graph("apigraph", redis_con)
-```
-
-For querying, URL should be provided first:
-
-```
-    url = input("url>>>")
-
-    return query(apigraph, url) # apigraph is vocab file provided by url.
-```
-
-The client takes the query as input, like:
-
-```
-    while True:
-        print("press exit to quit")
-        query = input(">>>")
-        if query == "exit":
-            break
-        elif query == "help":
-            help() # provide querying format
-        else:
-            print(facades.user_query(query))# query can be done with facades class
-```
-
-you can query as following querying formats:
-
-```
-    print("querying format")
-    print("Get all endpoints:- show endpoints")
-    print("Get all class_endpoints:- show classEndpoints")
-    print("Get all collection_endpoints:- show collectionEndpoints")
-    print("Get all members of collection_endpoint:-",
-          "show <collection_endpoint> members")
-    print("Get all properties of objects:-",
-          "show objects<endpoint_type> properties")
-    print("Get all properties of any member:-",
-          "show object<id_of_member> properties ")
-    print("Get all classes properties:-show class<class_endpoint> properties")
-    print("Get data with compare properties:-",
-          "show <key> <value> and/or <key1> <value1>")
-    print("Get data by using both opeartions(and,or)",
-          " you should use brackets like:-",
-          "show model xyz and (name Drone1 or name Drone2)",
-          "or, show <key> <value> and (<key> <value> or <key> <value>)")
-
-```
-
-Query test can be done like this:
-
-```
-    check_data = [['p.id', 'p.operations', 'p.properties', 'p.type'],
-                      ['vocab:EntryPoint/Location', 
-                       "['POST'", "'PUT'", "'GET']", 
-                       "['Location']", 'Location']]
-    query = "show classEndpoints"
-    self.assertEqual(data,check_data) #data is data retrieve from the Redis.
-```
-
-For more detail take a look at [wiki file](https://github.com/HTTP-APIs/http-apis.github.io/blob/master/hydra-agent-redis-graph.md)
+**Setup hydrus**
+Since this is an API Client, we need an appropriate Hydra Server to query to. To setup a localhost follow the instructions at https://github.com/HTTP-APIs/hydrus#demo. You might want to run `hydrus serve --no-auth` to skip setting up headers.
 
 #### Agent package 
-To use the Agent as a package you can simply do something like:
+After installing the Agent and running Redis, [as per instructions above](https://github.com/HTTP-APIs/hydra-python-agent/#user-content-installation), you can do something like:
 
 ```
 from hydra_agent.agent import Agent 
 
-agent = Agent("http://localhost:8080/serverapi")
-agent.get("http://localhost:8080/serverapi/DroneCollection/123-123-123-123")
+agent = Agent("http://localhost:8080/serverapi/") # <- hydrus Server URL 
+agent.get("http://localhost:8080/serverapi/DroneCollection/")
 ```
 
 The agent supports GET, PUT, POST or DELETE:
 
-- GET - used to READ resources or collections
-- PUT - used to CREATE new resources in the Server
-- POST - used to UPDATE resources in the Server
-- DELETE - used to DELETE resources in the Server
+- **GET** - used to READ resources or collections
+- **PUT** - used to CREATE new resources in the Server
+- **POST** - used to UPDATE resources in the Server
+- **DELETE** - used to DELETE resources in the Server
 
-To GET a existing resource you should:
+**To GET** a existing resource you should:
 ```
 agent.get("http://localhost:8080/serverapi/<CollectionType>/<Resource-ID>")
 agent.get("http://localhost:8080/serverapi/<CollectionType>/")
 ```
 
-To PUT a new resource you should:
+**To PUT** a new resource you should:
 ```
 new_resource = {"@type": "Drone", "name": "Drone 1", "model": "Model S", ...}
 agent.put("http://localhost:8080/serverapi/<CollectionType>/", new_resource)
 ```
 
-To UPDATE a resource you should:
+**To UPDATE** a resource you should:
 ```
 existing_resource["name"] = "Updated Name"
 agent.post("http://localhost:8080/serverapi/<CollectionType>/<Resource-ID>", existing_resource)
 ```
 
-To DELETE a resource you should:
+**To DELETE** a resource you should:
 ```
 agent.delete("http://localhost:8080/serverapi/<CollectionType>/<Resource-ID>")
 ```
 
 More than that, Agent extends Session from https://2.python-requests.org/en/master/api/#request-sessions, so all methods like auth, cookies, headers and so on can also be used.
 
-#### Querying Redis
-Reference can be found here: https://oss.redislabs.com/redisgraph/commands/
-
-Entity structure: alias:label {filters}.
-
-Example of MATCH:
-(a:actor)-[:act]->(m:movie {title:"straight outta compton"})
-
-GRAPH.QUERY apigraph "MATCH (p) RETURN p" 
-
-Internal Hydra Python Agent naming:
-
-Labels:
-
-- collection, classes - macro labels
-- objects<ObjectType> - for members
-- object<ObjectID> - for resources
+### Natural-language-like Command Line Tool
+If you've followed the [installation](#installation) instructions you can run: 
 
-Aliases:
+    python hydra_agent/querying_mechanism.py
 
-Alias for collection example:
-<ObjectType>Collection
-DroneCollection
+Another alternative to run the CLT is using docker componse. To run **both Redis server and the client**(stop any Redis instance before), you can run the command:
+    
+        docker-compose run client
 
-Alias for collection member:
-<ObjectType><ObjectID>
-Dronea9d6f083-79dc-48e2-9e4b-fd5e9fc849ab
+To query you should provide a hydrus URL first:
 
-To get all nodes from the Graph:
 ```
-GRAPH.QUERY apigraph "MATCH (p) RETURN p" 
+     url>>> http://localhost:8080/serverapi/ 
+
 ```
 
-Get all nodes and filter by label:
-```
-GRAPH.QUERY apigraph "MATCH (p:collection) RETURN p" 
-```
+**Obs.: If failing to connect to localhost** running the Agent via Docker, head to [issue #104](https://github.com/HTTP-APIs/hydra-python-agent/issues/104#issuecomment-497381440).
 
-Get all nodes and filter by label:
-```'
-GRAPH.QUERY apigraph "MATCH (p) WHERE(p.id = '/serverapi/DroneCollection/72b53615-a480-4920-b126-4d1e1e107dc6') RETURN p" 
-```
+- **Natural Language querying format**
 
-To read all the edges of the graph
-```
-GRAPH.QUERY apigraph "MATCH ()-[r]->() RETURN type(r)"
-```
+Run help inside the CLT to get the querying format.
 
-To read all the edges connected to a node
-```
-GRAPH.QUERY apigraph "MATCH (p)-[r]->() WHERE p.type = 'DroneCollection' RETURN type(r)"
-```
+        >>>help # it will provide the querying format
 
-Creating Edges between existing Nodes(Ref: https://github.com/RedisGraph/redisgraph-py/issues/16):
-*This is not available yet on the oficial doc*
-```
-MATCH (f:%s{%s:'%s'}), (t:%s{%s:'%s'}) CREATE (f)-[:in]->(t)
-GRAPH.QUERY apigraph "MATCH (s:collection {type:'DroneCollection'} ), (d:objectsDrone {id:'/serverapi/DroneCollection/ea7e438e-a93d-436d-a7e9-994c13d49dc0'} ) CREATE (s)-[:has_Drone]->(d)"
-```
+You can query the server with the following format:
 
-To create a node:
-```
-GRAPH.QUERY apigraph "CREATE (Droneea7e438e-a93d-436d-a7e9-994c13d49dc0:objectsDrone {@id: '/serverapi/DroneCollection/a9d6f083-79dc-48e2-9e4b-fd5e9fc849ab', @type: 'Drone', model: 'Ultra Model S')"
 
-```
+> Get all endpoints:- **show endpoints**
+Get all class_endpoints:- **show classEndpoints**
+Get all collection_endpoints:- **show collectionEndpoints**
+Get all members of collection_endpoint:- **show < collection_endpoint > members**
+Get all properties of objects:- **show objects< endpoint_type > properties**
+Get all properties of any member:- **show object< id_of_member > properties **
+Get all classes properties:- **show class< class_endpoint > properties**
+Get data with compare properties:- **show < key > < value > and/or < key1 > < value1 >**
+Get data by using both opeartions(and,or)  you should use brackets like:- **show model xyz and (name Drone1 or name Drone2) or, show < key > < value > and (< key > < value > or < key > < value >)**
 
-To delete a node:
-```
-GRAPH.QUERY apigraph "MATCH (p) WHERE (p.id = '/serverapi/DroneCollection/2') DELETE p"
+For more detail take a look at [wiki file](https://github.com/HTTP-APIs/http-apis.github.io/blob/master/hydra-agent-redis-graph.md)
 
-```
+### Missing bits at the moment
+* For the Web GUI there's a list of [future enhacements here](https://github.com/HTTP-APIs/hydra-python-agent-gui/issues/3).
+* For now the Natural language CLT it is a proof-of-concept, only `GET` functionality
 
 References
 ----------

diff --git a/docker-compose.yaml b/docker-compose.yaml
@@ -14,6 +14,6 @@ services:
         environment:
             - REDIS_HOST=redis_db
     db:
-        image: redislabs/redisgraph:edge
+        image: redislabs/redisgraph:2.0-edge
         ports:
             - "6379:6379"
-Original file line number
+Diff line change
@@ Expand Up / @@ -3,7 +3,7 @@ services: @@
       - docker
     before_script:
-      - docker run -d -p 6379:6379 -it --rm --name redisgraph redislabs/redisgraph:edge
+      - docker run -d -p 6379:6379 -it --rm --name redisgraph redislabs/redisgraph:2.0-edge
     python:
       - "3.5"
@@ Expand Down @@