We are proud to have launched our first competition on Kaggle!
The primary challenge of this competition is to develop an AI tool or method that can accurately extract questionnaire questions from documents, primarily PDFs.
This competition offers a unique opportunity for participants to contribute to the field of natural language processing and document analysis as well as open source for social science while developing solutions that have real-world applications. We encourage participants to think creatively, leverage available resources, and push the boundaries of current technologies.
Requirements: Python 3.10 or greater
Create an account on Kaggle.
Install Kaggle on your computer:
pip install kaggle
On the Kaggle website, download your kaggle.json
file and put it in your home folder under .kaggle/kaggle.json
.
Download and unzip the competition data:
kaggle competitions download -c harmony-pdf-and-word-questionnaires-extract
unzip harmony-pdf-and-word-questionnaires-extract.zip
To generate predictions for the training data and write to train_predictions.csv:
python create_sample_submission.py train
To evaluate the train predictions:
python evaluate_train_results.py
To modify the prediction logic or inject your own model, you can edit the function dummy_extract_questions
.
To generate predictions for the test data and write to submission.csv:
python create_sample_submission.py test
kaggle competitions submit -c harmony-pdf-and-word-questionnaires-extract -f submission.csv -m "Message"
New Feature in Development: Enhancing Harmony with User-Centred Discovery At Harmony, we’re working on a new feature aimed at making data exploration and analysis even more efficient and intuitive. To ensure this feature addresses real needs, we’re conducting co-design sessions with researchers, data-managers and other users, allowing us to develop a tool that solves real-life user requirements. The Co-Design Approach: Co-design allows us to build this feature with direct input from those who will use it most.
We have a few more issues that have been added to the issue trackers. If you are new and would like to make a pull request in either the Python or R libraries feel free to pick these up - they should be quite small. Easy issues in Python library We would like to expose the “between instrument matches” and the “negation” switches in the Python library and then from the API side.