development

Contributing to Harmony

Please select all the ways you would like to hear from Harmony project:

Contributing to Harmony

Contribute to the Harmony open source NLP project

Are you a scientist, researcher, data wrangler, or language maestro? Harmony needs YOU! We’re always looking for talented individuals to join our team.

  • Contribute to our open-source code: Whether you’re a seasoned developer or a curious newbie, your contributions are valued.
  • Join the conversation: Join our Discord server, or follow us on Twitter, LinkedIn, and other social media channels.

Getting started

Participating in an open source project can be very rewarding. Read more about it here!

Please familiarise yourself with Git. You can fork Harmony and make a pull request any time! We’re glad to have your contribution.

Video of our orientation session on how to contribute to Harmony.

Issues and bug reports

First, do a quick search to see if the issue has already been reported. If so, it’s often better to just leave a comment on an existing issue, rather than creating a new one. Old issues also often include helpful tips and solutions to common problems. You should also check the troubleshooting guide to see if your problem is already listed there.

If you’re looking for help with your code, consider posting a question on the GitHub Discussions board. Please understand that we won’t be able to provide individual support via email. We also believe that help is much more valuable if it’s shared publicly, so that more people can benefit from it.

Make your first contribution

There are lots of ways you can contribute to Harmony! You can work on code, improve the API, or add code examples.

  • Write code
  • Improve unit tests or integration tests
  • Add new functionality to Harmony
  • Improve Harmony’s documentation
  • Add integrations to other LLMs or LLM providers such as OpenAI, IBM, or similar
  • Add integrations from your website to Harmony
  • Publicise Harmony in web forums such as Reddit, HuggingFace forum, Quora, or similar
  • Create example notebooks, such as Jupyter Notebook, RStudio, or Google Colab
  • Investigate bugs and issues in Harmony
  • Review and comment on pull requests
  • Cite Harmony in your blogs, papers, and articles
  • Talk about Harmony on social media. Don’t forget to tag us on Twitter @harmony_data, Instagram @harmonydata, Facebook @harmonydata, LinkedIn @Harmony, and YouTube @harmonydata!
  • Starring and forking Harmony on Github!

Where do we need help in Harmony?

In particular, the PDF extraction (converting PDFs to structured questionnaire items) is very hard and we have a separate Github repo with examples here: https://github.com/harmonydata/pdf-questionnaire-extraction

We are planning on running a hackathon focused on this aspect of the tool.

Also, other initiatives that could be really useful include:

  • Better handling of active vs passive voice in questionnaire items
  • Allowing Harmony to switch LLMs
  • Integration with other websites and tools
  • An h-score: a similarity measure between instruments

We started to make a new repo with training data to improve the PDF data processing. This repo has manually annotated training data: https://github.com/harmonydata/pdf-questionnaire-extraction

One issue that would be really helpful would be, to handle active and passive voice, such as “child is bullied” vs “child bullies others” - these are harmonised as close together whereas they should be far apart.

Maybe a small task is easier to start with. Can you see any obvious bugs that you’d like to pick up?

We’ve been looking at integrations with other data repositories. One way forward is to make an npm package other sites can install that makes it easy for them to send data over to the app. Or possibly just to interact with the API directly.

Raising issues and the issue tracker

The issue list is in the Github repository. You can view the open issues, pick one to fix, or raise your own issue. Even if you’re not a coder, feel free to raise an issue.

Coding Harmony

Harmony is mostly coded in Python. We use Pycharm IDE by JetBrains. Please ensure you are familiar with Python, HuggingFace, and FastAPI, or Javascript and React if you want to work on the front end.

Please make sure all code you commit is linted using the Pycharm default linter. If you use a different one (such as VS Code’s linter, or pylint), this will make the code history hard to follow, so please be consistent.

Unit tests and code stability

Harmony uses the pytest framework for testing. For more info on this, see the pytest documentation. To be interpreted and run, all test files and test functions need to be prefixed with test_.

The Harmony Python library https://github.com/harmonydata/harmony is the core Harmony functionality. Most of the logic is in this repo. This repo has unit tests which run automatically on commits to main.

However, the Harmony API repo https://github.com/harmonydata/harmonyapi uses the Harmony Python library as a submodule. When you update the Python library, please run the unit tests and integration tests in the API repo to check nothing is broken - including the Selenium tests which test the browser app end to end. You will need to install Selenium to run the tests.

Since the API repo includes the Python library as a submodule, when you update the Python library, you will need to update the submodule (in the harmonyapi repo, cd into the submodule folder and do git pull, then cd out and do git add, commit and push). We recommend you familiarise yourself with Git submodules.

Finally, the app repo https://github.com/harmonydata/app is the React front end. Please check you can run this repo locally also before you start contributing. To point the front end repo to a local copy of your API repo, please change the file .env to point to http://localhost:8000.

Pull requests

If you’d like to contribute to this project, you can contact us at https://harmonydata.ac.uk/ or make a pull request on our Github repository. You can also raise an issue.

Related Posts

Mastering Survey Data Analysis with Harmony - A Smart Guide

Mastering Survey Data Analysis with Harmony - A Smart Guide

Comparing Apples to Oranges: How to Analyse Two Completely Different Surveys So, you’ve created a survey (or multiple surveys), convinced people to take part and answer your questions, and have received the results. Great job! Yet, the real test comes when it’s time to make sense of these survey findings. Today, we’ll take a look at why you need surveys, how to design effective surveys, how to analyse survey data, how to analyse survey data with multiple responses, and how to do data harmonisation when you’re dealing with multiple surveys.

Data harmonisation tools and frameworks in 2024

Data harmonisation tools and frameworks in 2024

Data Harmonisation Tools – Overview Organisations typically collect data from multiple sources and for many different reasons. This data comes in various forms and formats – for example, it may be coming from market research, customer research or inter-organisation departments. Data harmonisation, an advanced technique used to make sense of all the raw data collected and uses for research purposes, becomes necessary in this case, but unless it is incorporated effectively, organisations might miss the full, holistic view of business performance that they wish to gain.

Signup to our newsletter

The latest news on data harmonisation project.

Please select all the ways you would like to hear from Harmony project:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website. We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices.