The VILMA database is a collection of molecular databases with relevance to the VILMA project. The database is implemented as a document-oriented NoSQL MongoDB server instance. The MongoDB instance hosts multiple databases where each individual database has been constructed from some molecular dataset(s). Each individual database in turn is divided into one or more partitions called collections. The NoSQL nature of the database means there is no pre-defined schema i.e. strict structure for any of the individual collections, but rather each of them is a data store of BSON (Binary JSON) documents.

Thus the general hierarchy of the data can be described as: Instance (the host server) → Database (constructed from some dataset(s)) → Collection (some partition of the dataset(s)) → Document (each individual data element). Each document has one or more key-value pairs referred to as fields. In general it can be assumed that all of the documents within the same collection have the same fields, but in theory this is not guaranteed or enforced by default by the database engine.

The following will describe the general steps for connecting to the databases within the VILMA MongoDB instance and how to access the data within specific collections with the python client.

This database has been constructed by parsing the GeckoQ and its addendum datasets.

This database has been constructed by parsing the filtered GeckoAP datasets.

In order to access the MongoDB instance you need credentials in the form a connection string. It has the general structure of: mongodb://<username>:<password>@<hostname>:<port>. The :<port> segment is optional. If omitted, the driver defaults to port 27017.

Make sure the connection string provided to you follows this format. By default the credentials given will be read-only, meaning you can query and retrieve data but cannot modify or delete existing records.

For security, never hard-code your connection string in your source code. Store it as an environment variable in a .env file at your project root:

MONGO_URI=mongodb://<username>:<password>@<hostname>

To use these credentials in Python, use the python-dotenv library. To install it with pip, use pip install python-dotenv. Then to access the environment variable do:

import os
from dotenv import load_dotenv

load_dotenv()
uri = os.getenv('MONGO_URI')

MongoDB Compass provides a Graphical User Interface (GUI) while MongoDB Shell (not to be confused with MongoDB CLI) provides a terminal-based interface for data interaction. Both support direct data access via queries and advanced data processing via aggregation pipelines.

MongoDB Compass:

• Download: MongoDB Compass Download
• Connection Guide: How to Connect
• Official Documentation: Compass Docs

MongoDB Shell:

• Download: MongoDB Shell Download
• Connection Guide: How to Connect
• Official Documentation: Shell Docs

You can connect to the MongoDB instance within Python by using the PyMongo library. To install it with pip, use pip install pymongo. Then, to create a connection and accessing a specific collection within a specific database do the following:

from pymongo import MongoClient

db_name = "database_name"
collection_name = "collection_name"

client = MongoClient(uri)
db = client[db_name]
collection = db[collection_name]

Here the variable uri is connection string read from the environment variable as described under the section Access Credentials.

If you are not sure what the available database names are within the MongoDB instance, you can list them with the following:

print(client.list_database_names())

If you are not sure what the available collection names are within a specific, you can list them with the following:

print(db.list_collection_names())

The rest of the documentation will describe how typical tasks are achieved using PyMongo.

• Full Official Documentation: PyMongo Docs

A general query can be defined as a dictionary where the keys correspond to fields within the collection and values to dictionaries of restrictions on those fields. Each query can include multiple fields and each field can have multiple restrictions. The restriction dictionary keys are query operators and values the operands of the operators.

To run the query we use the find method applied on a collection. The output will be a cursor that can be iterated over to retrieve documents matching the query as dictionaries.

These operators allow you to filter documents based on value comparisons.

These operators allow you to combine multiple query conditions.

{"num_conformers": {"$gt": 100, "$lt": 500}}

{"$and": [{"num_conformers": {"$gt": 100}}, {"num_conformers": {"$lt": 500}}]}

By default the find method returns all the fields of the matching documents. We can however specify which fields should be included in or excluded from the resulting cursor to save bandwidth and memory. This called projecting the data.

The projection is a dictionary where the values are the names of the fields and values 1 for fields to include and 0 for fields to exclude. This dictionary is then given as the second argument for the find method.

1. The simplest query is the empty dictionary which will return all the documents within the collection:

results = collection.find({})

2. The following query finds all the documents within the collection, but projects to include only the fields molecule_index and valid_flexibility:

projection = {
    "_id": 0,
    "molecule_index": 1,
    "valid_flexibility": 1
}

results = collection.find({}, projection)

3. The following query finds all the documents with the field num_conformers within the range $[100, 500]$:

query = {
    "num_conformers": {"$gte": 100, "$lte": 500}
}

results = collection.find(query)

4. The following query finds all the documents with the field source not equal to "addendum", field isomer_count equal to $1$, and field valid_flexibility greater than $0$, and projects to include only the fields molecule_index, valid_flexibility, and num_conformers:

query = {
    "source": {"$ne": "addendum"},
    "isomer_count": {"$eq": 1},
    "valid_flexibility" : {"$gt": 0}
}

projection = {
    "_id": 0,
    "molecule_index": 1,
    "valid_flexibility": 1,
    "num_conformers": 1
}

results = collection.find(query, projection)