BlueGenes is a new user interface to InterMine utilising its extensive web services combined with a modern web application stack, to provide a more interactive experience for exploring and analysing your data. In contrast with the previous interface, BlueGenes is separate from the InterMine server, increasing the ease of further development to both.

The new interface provides the same feature set you’re already familiar with, wrapped in a new design offering an enhanced user experience. In addition, it allows you to quickly change the InterMine server you’re connected to, to any other instance in your organisation or that is available through the InterMine Registry. While the old webapp acted as a front-end to a single InterMine, BlueGenes can be configured to serve multiple InterMine servers.

A new Tool API for adding custom visualisations and analyses tools to your InterMine has also been introduced. This allows you to create your own tiny JavaScript apps (or adapt the ones we have developed for HumanMine and FlyMine) for your InterMine data, which BlueGenes will display for list analysis, query results and/or report pages, depending on what you specify it supports.

While this is the first major release, we still have lots more coming. Some admin tasks, notably creating templates, is not available in BlueGenes yet so you will have to access the legacy webapp for this. The web services have yet to be separated from the legacy webapp, so it will continue to be available for a while (if you wish to wait with switching to BlueGenes, we recommend using the latest InterMine 4.x release instead).

This new release requires InterMine 5.0.0. To upgrade to InterMine 5.0.0 please read the upgrade instructions carefully.

InterMine 5.0.0 release

This releases contains new web services to support BlueGenes’s new features, in particular:

• Login using OAuth2 (e.g. Google, Elixir)
• Upgrading lists
• Generate external links
• Submit feedback
• Reset password

This release includes the Frictionless Data Package specifications implemented by Nikhil Vats. Please see the documentation here for more details.

All the InterMine artifacts have been published on the Maven Central repo instead of Bintray’s JCenter due to the recent jcenter/bintray shutdown announcement.

For more details about this release, please see the release notes.

Documentation

The user documentation has been adapted to the new interface by Hanan (Outreachy intern) and is available here.

The InterMine Server documentation (aka dev documentation) has been revised by Mardhiyah (Outreachy intern) and is available here.

FlyMine and HumanMine

FlyMine has been updated to use BlueGenes. If you use Firefox, you might experience an issue due to the browser cache, in that case you need to delete the cache. The legacy user interfaces is still available here.

HumanMine will be updated soon.

Thanks to …

… all our users/contributors for their support, feedback and help. Thanks to Joshua Heimbach and Yo Yehudi for their great and essential past contributions to this project. Thanks to our excellent interns who have directly contributed to BlueGenes; Akshat for the 2019 GSoC project Javascript Data Visualisations, Sakshi Srivastava for the 2020 Outreachy mid-year project Javascript Data Visualisations, Roshni Prajapati for the 2020 Outreachy mid-year project Improve UI & UX of BlueGenes, and Hanan Younes for the 2020 Outreachy end of year project Create InterMine user training docs.

Contact us

If you have any questions or concerns about any of these changes, please do not hesitate to contact us.

We are pleased to announce the new InterMine release 4.3.0.
It includes a few improvements and bug fixes.
This is a non-disruptive release.

Improvements and bug fixes

1. Added a new generic obo source which can be configured in project.xml (Sam Hokin)
2. Removed the gretty plugin from testmine which now uses gradle 4.9
3. Removed duplication of so.obo file
4. Fixed the Google Charts API which broke various things, including the humanmine expression viewers
5. Updated Google auth and userinfo endopoints
6. Fixed the NullPointerException in the report page caused by an empty value in the field key used to generate the permanent URI (Share button)
7. Fixed the home page loading when bioschemas markup are enabled (markup.webpages.enable=true) and JAVA 11 is used
8. Fixed the webservice query/results when the format is jsonobjects and the root class is a simple object – with no id (Sam Hokin)

See release notes for detailed information.

Upcoming releases

For more information about the upcoming releases, please visit the InterMine roadmap here.

Hi Mardhiyah! We’re really excited to have you on board as part of the team this winter. Can you introduce yourself? 

Hi InterMine team! I am also very excited to be working with you guys during this winter. I am a recent Electronic and Electrical Engineering graduate. I am a python developer and an aspiring machine learning engineer with particular interests in computer vision and natural language processing. I enjoy being in intellectually stimulating environments as this satisfies my hunger for solving problems of varying nature.

I also enjoy watching movies, videos of pandas and playing (probably an unhealthy amount ) of scrabble.

What interested you about Outreachy with InterMine?

Applying to outreachy was my first experience with open source, so I wanted to start from contributing to a documentation related project. However, after starting with the InterMine project and interacting with the very accommodating community, I was instantly attached to this project and team. I must say that the community made my first experience with contributing to open source a very memorable and easy one and I am therefore looking forward to spending the next three months working with the team.

Tell us about the project you’re planning to do for InterMine this winter.

The project is “Review, update, and integrate InterMine developer documentation”. I intend to evaluate different technical documentation frameworks that support markdown and versioning as an alternative to the current documentation that uses Sphinx and Read the Docs. I also plan on reviewing the current documentation for issues and implementing solutions, converting files written in reStructuredText to markdown format as well as integrate the two documentation sources.

Are there any challenges you anticipate for your project? How do you plan to overcome them?

The main challenges I’m anticipating are time and organization. However, I have come up with a detailed timeline, highlighting when tasks should be started and completed and a spreadsheet to keep track of converted and reviewed files. 

Share a meme or gif that represents your project

We are thankful to a Wellcome Trust Diversity and Inclusion Grant for funding three of our interns, Outreachy for matching this by funding two interns, and our main Wellcome grant for making it possible to fund two more. This brings us up to a count of seven interns, the highest number that we’ve had the pleasure of working with yet at one time! This wouldn’t have been possible without help from our external mentors — Akshat Bhargava, Aman Dwivedi, Ankur Kumar, Asher Pasha and Nikhil Vats — whom will all be receiving a small prize as a token of our gratitude.

During the internship period, we had to say goodbye to our valued InterMine team member for 5 years, Yo Yehudi, who drove our internship scheme to its current state and has left to become the Technical Lead for Open Source at the Wellcome Trust. Thanks to the efforts of our mentors for their support of our interns, and to Rachel for running the final presentation call, we were able to provide a closing to the internship period we can all be proud of.

This is the last week of the Outreachy Internship. From filling up the initial application form just based on the gut feeling even though I thought I lacked technical skills to apply for an internship, picking up intermine org to getting selected, the memories seem a bit fuzzy.

If someone would have asked me before applying to Outreachy whether I’d be able to get selected and successfully complete the internship, my answer would probably have been a no.

In a mere span of a few months, I feel like I have come quite a long way in terms of my confidence, approach to new problems and skills. I’ll reflect on three parts in this blog – my before and after thoughts of the internship, how Outreachy helped me grow and progress of my project.

Apart from my little stint at Hacktoberfest 2019, I did not have any quotable experience working with OpenSource. I feel lucky to land in such an encouraging org. I have had trouble thinking about communicating during the project as I have mostly just communicated in my mother tongue so understanding different accents or fear of mis-pronouncing words, mis-form sentences are there. But I never saw any judgement from anyone based on how correct my communication skills were. The only important thing is to express yourself. This really helped my self confidence. Before applying to Outreachy, I thought it is for people who have good open source experience and skill set, who have worked on reasonably big projects and are able to carry on the project completely independently. Now, I would say Outreachy is for anyone having basic skills and trying to learn new things. The collaborative environment of open source would surprise you.

I think Outreachy has helped me grow in a multitude of ways. It has definitely boosted my self confidence. It taught me working in collaborative projects and how it is ok to ask for help when stuck and not feel bad about it. Amongst technical skills, I have learnt about docker, docker-py, github actions, picked up more info about bash, python along the way.

The concept of having a mentor (actually two mentors!) is great. I have a background in research where most of the time I am all on my own to move the project forward (codewise) if stuck. Discussions with the advisor are mainly on the theoretical and experiment part. I am very grateful for my mentors being patient and guiding me during the internship.

I have definitely deviated from my initial project proposal. Some old tasks were removed paving the way for picking new tasks along the way. I think majorly the task of setting up wizard and configuration was not touched because they are not ready to be integrated. The task of running any mine (and not just mines based on biotestmine) would still require a bit of tussle. So there are definitely many open tasks requiring contribution.

I have got a taste of open source from Outreachy and hope to carry it forward. 

All good things come to an end eventually, but the next experience awaits

Anonymous

In this blog, I reflect on my academic journey so far and what I would like to do after the Outreachy internship ends.

I have done my schooling in Rajasthan, India. After appearing for RPET (Rajasthan Pre Engineering Test), a popular entrance exam for engineering college admissions in Rajasthan, I got admitted in Information Technology branch in Govt. Women College in Rajasthan which is one of the best Women only Colleges in India.

I had worked on some projects in college but exposure to diverse technologies was very much less. I felt compelled to pursue a master’s degree to both increase job opportunities and dive deeper in one of the domains in computer science.

After completing the BTech I worked in a startup, After that I got admission in IIIT Hyderabad, India in 2017 as a part-time Mtech student in computer science. As my interest developed, I talked to different research advisors for exploring my knowledge in research and converted to full-time MS student in data science under Prof P. Krishna Reddy. My MS would tentatively finish around May/June 2021.

I am looking towards joining the industry in an engineering role. I am interested both in data science and software development. My college organizes campus placement sessions around December every year. After Outreachy, I am planning to polish my data structures and algorithms knowledge and sit for campus placement.

I feel that after coming to IIIT, I have gained a lot of skills. I have worked on projects in distributed systems and machine learning. I was head teaching assistant for the Distributed Systems course and managed making assignments and paper evaluations for a class of around 150 students.

I am open to learning new skills for advancing my career. I am also open for remote jobs. I can communicate in Hindi and English language.

Github: https://github.com/22PoojaGaur
Linkedin: https://www.linkedin.com/in/pooja-gaur-b5848048/

To everybody reading this. Please feel free to contact me in case you want to ask anything or you feel that I could be a part of your organization!

The only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle.

Steve Jobs

I started the Outreachy internship with Intermine on 19th May 2020. As I write this blog, I have covered a little over half of this 3 month journey. Through this blog, I will look back in retrospect on what has been done, what goals have been achieved, what goals have been changed, what remains in the coming half of Outreachy. Let’s get started!

I am working on the Intermine boot project in the Intermine organization. As part of the internship, the goal is to improve the intermine boot functionality for local instance setup and integrate it with cloud API for other features. The intermine boot project is at a very early stage of development. Let’s break down the tasks in few big chunks:
– Implement versioned mine data uploading to cloud storage.
– Create Continuous Integration setup case
– Add wizard and configurator to get custom configuration from project
– Use all of the above to orchestrate docker containers for usage.

As with all projects, you can not plan everything beforehand and requirements change/emerge as the project evolves. Right from the start, one additional task was updated to move the intermine_boot project from using docker-compose file to use docker-py for setting up intermine instance. Another extra work was to add docker-intermine-gradle as a submodule to the project.

The wizard and configurator are part of the Intermine Cloud project. They are not at a state where we can integrate them intermine_boot right now. So, they’ve been moved back in the project and that task would be pursued if they manage to be at a state so that they can be integrated. In place of that, my mentor has added other tasks which are needed to improve the usability of intermine_boot.

I have submitted 6 pull requests for the project which cover some of the tasks mentioned above. I am fairly happy with my progress. There were roadblocks which led to slower progress than I would have initially assumed but I think I am getting better equipped to handle the coming half of the project.

I feel very happy that I got accepted to such an amazing community and got a great mentor!

Now, buckling up for the next half of the internship to get stuff done!!

Start by doing what’s necessary, then do what’s possible; and suddenly you are doing the impossible 

Saint Francis of Assissi

Data is of paramount importance in research works. In biological research domains, there are multiple research communities working and generating new biological datasets for DNA, yeast, mouses etc etc. At the same time, there are many researchers who need to work with these datasets for their research projects.

One way to share data may be to just hand over archived datasets. In this case, there are numerous problems like, how can you understand the data format, how do you clean this data in case of any inconsistency, how do you search through this data, how do you integrate this data with different datasets, how do you store huge data.

Intermine is a biological data warehouse which aims to resolve these issues and make accessing data easier for researchers. Once a dataset is added to intermine, users can perform complex queries over it to get the required information.

There are different intermines for different types of data like FlyMine, YeastMine, HumanMine, WormMine. The intermine project is open source and it allows research organizations to set up intermine instances dedicated to their datasets.

An intermine instance provides both web app and web service where you can host data and clients can make queries to get integrated biological data. Now that we have covered basics, let’s move towards why the project I am working on becomes relevant!

Setting up your own instance of intermine is a time consuming and complex process requiring a fair amount of Linux administration skills. We would want to make this process easier so that people with very little programming knowledge can do it. Intermine cloud project attempts to solve this and lower the barrier of running an intermine instance.

Intermine Cloud is composed of three main parts – wizard, configurator and compose. The wizard provides an easy way for setting custom configuration for the new intermine instance. The configurator is the backend of the wizard which creates necessary configuration files required to build the intermine instance. Once an intermine instance is built, the compose handles deploying and managing intermine instances on the cloud.

At times, a user may want to set up the intermine instance locally to see how the project will look or while he is trying to make some customizations to extend intermine for the different use cases. Or if he wants to host the intermine instance on his own servers. That’s where the Intermine Boot project comes in.

Intermine Boot is a command line tool which aims to allow users to easily setup local intermine instances inside docker containers, upload data archives to the cloud and other functionalities to make the convenience features for users.

Let’s understand the use case with an example. Suppose as an end user, you get interested in intermine. You want to set up and host your intermine instance on your servers. You dig in the documentation, start setting up postgresql, gradle, perl, solr etc etc. Meanwhile, you are also polluting your system’s environment in case you are not using docker or any other virtualization. The intermine boot aims to make this process as easy as running few commands on terminal. Below is a meme version to explain the benefits in a funny way!

You can find the intermine boot at https://github.com/intermine/intermine_boot and all intermine org projects at https://github.com/intermine

This is enough introduction for the Intermine and Intermine boot. Feel free to dive in the project now, we have a lot of interesting things going on!

If you can’t explain it simply, you don’t understand it well enough.

– Albert Einstein

We are still working towards the production release of BlueGenes, at which point we can recommend it for future deployments over the legacy user interface. This recent year has brought with it a plethora of necessary technical improvements and bug fixes, along with new additions to bring the user interface towards feature parity with the current webapp. The following details the most visible changes to BlueGenes in the last release, which you can explore by updating your local instance or using the public BlueGenes instance.

Previous minor releases

There have been some notable changes in prior minor releases. As they haven’t been mentioned in a blog post, we will include them here.

BlueGenes related improvements

1. Added /login web service that merges the anonymous session with the user logged in.
2. Added /logout web service.
3. Added a new webservice to change the users’s password.
4. Updated the existing /lists webservice which allows modifying the list description.
5. Improvements on the Date type (to support CovidMine).

BlueGenes 0.10.0 will be released soon and announced in a separate blog.

FAIR related improvements

1. Simplified the webservice that generates Bioschemas markup for the report page.
2. Adopted DataRecord in the report page.
3. Added Gene, Protein markup in the report page.
4. Added BioChemEntity markup in the report page (only if configured).
5. Added the ontology licences to the obo converters.

Bug Fixes / Improvements

1. Added a new bio source to load ISA files in json format
2. Fixed organism short name generation (Ahmed Hafez)
3. Fixed a bug related to long fields in the report page (Asher Pasha)
4. Removed BioEntity.ontologyAnnotations because redundant (Sam Hokin)
5. Fixed src.data.dir.include (gff3 and xml) ans src.data.dir (intermine-items-xml-file)
6. UniProtFastaLoader works with organism names longer than 2 words (for example Severe acute respiratory syndrome coronavirus 2)

See release notes for detailed information.

Upcoming releases

For more information about the upcoming releases, please visit the InterMine Development Roadmap. More details on the roadmap here.