development

Harmony on Kaggle

Please select all the ways you would like to hear from Harmony project:

Harmony on Kaggle

Harmony launches on Kaggle!

We are proud to have launched our first competition on Kaggle!

The primary challenge of this competition is to develop a tool or method that can accurately extract questionnaire questions from documents, primarily PDFs.

This competition offers a unique opportunity for participants to contribute to the field of natural language processing and document analysis while developing solutions that have real-world applications. We encourage participants to think creatively, leverage available resources, and push the boundaries of current technologies.

Try Kaggle

Try your hand at our competition

Kaggle Github repo

Check out the Github repo associated with the Kaggle competition and the tagged PDF data

Entering the Kaggle competition

Requirements: Python 3.10 or greater

  1. Create an account on Kaggle.

  2. Install Kaggle on your computer:

pip install kaggle
  1. On the Kaggle website, download your kaggle.json file and put it in your home folder under .kaggle/kaggle.json.

  2. Download and unzip the competition data:

kaggle competitions download -c harmony-pdf-and-word-questionnaires-extract
unzip harmony-pdf-and-word-questionnaires-extract.zip 
  1. Run create_sample_submission.py in the folder containing your data to create your train and test predictions:

To generate predictions for the training data and write to train_predictions.csv:

python create_sample_submission.py train

To evaluate the train predictions:

python evaluate_train_results.py
  1. To modify the prediction logic or inject your own model, you can edit the function dummy_extract_questions.

  2. To generate predictions for the test data and write to submission.csv:

python create_sample_submission.py test
  1. Submit your CSV file to Kaggle
kaggle competitions submit -c harmony-pdf-and-word-questionnaires-extract -f submission.csv -m "Message"

Related Posts

Harmony issue tracker

Harmony issue tracker

Tracking Harmony issues and raising feedback Have you found a problem with Harmony? You can use the links on the Harmony homepage at harmonydata.ac.uk - you can see there are buttons titled Raise NLP issue and Raise UI issue, which take you to the process for logging issues in Github. The issues remain in the issue trackers and can be assigned to people either under “Assignee” or in the comments:

Developer guide

Developer guide

Git and GitHub workflow The preferred workflow for contributing to Harmony’s repository is to fork the main repository on GitHub, clone, and develop on a new branch. Please read our general guide about contributing to Harmony. Fork the main project repository by clicking on the ‘Fork’ button near the top right of the page. This creates a copy of the code under your GitHub user account. For more details on how to fork a repository see this guide.

Signup to our newsletter

The latest news on data harmonisation project.

Please select all the ways you would like to hear from Harmony project:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website. We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices.