We are proud to have launched our first competition on Kaggle!
The primary challenge of this competition is to develop a tool or method that can accurately extract questionnaire questions from documents, primarily PDFs.
This competition offers a unique opportunity for participants to contribute to the field of natural language processing and document analysis while developing solutions that have real-world applications. We encourage participants to think creatively, leverage available resources, and push the boundaries of current technologies.
Requirements: Python 3.10 or greater
Create an account on Kaggle.
Install Kaggle on your computer:
pip install kaggle
On the Kaggle website, download your kaggle.json file and put it in your home folder under .kaggle/kaggle.json.
Download and unzip the competition data:
kaggle competitions download -c harmony-pdf-and-word-questionnaires-extract
unzip harmony-pdf-and-word-questionnaires-extract.zip
To generate predictions for the training data and write to train_predictions.csv:
python create_sample_submission.py train
To evaluate the train predictions:
python evaluate_train_results.py
To modify the prediction logic or inject your own model, you can edit the function dummy_extract_questions.
To generate predictions for the test data and write to submission.csv:
python create_sample_submission.py test
kaggle competitions submit -c harmony-pdf-and-word-questionnaires-extract -f submission.csv -m "Message"
Sending data from another website to Harmony using Javascript We have exposed functionality for external websites to integrate with Harmony and add an “import to Harmony” button, either generated in Javascript or in Python. Create an Instrument object with at least an instrument_name and questions property in JSON - the questions must have a question_no and question_text properties eg: { "instrument_name": "Smoking behaviour", "questions": [ { "question_no": "1", "question_text": "Do you currently smoke or have you ever smoked?
Harmony at PyData London - 86th Meetup Update: you can download the slides from the presentation here Topic: NLP and generative models for psychology research Thomas Wood will present our work on Harmony, harmonydata.ac.uk, which is a free online tool that uses generative AI and LLMs to help psychologists analyse datasets. It uses Python, Pandas and HuggingFace Sentence Transformers to find similarities between questionnaires. Psychologists and social scientists often have to match items in different questionnaires, such as “I often feel anxious” and “Feeling nervous, anxious or afraid”.