pybool_ir.experiments

Provides all the functionality for performing experiments using pybool_ir. It includes classes for execuring queries, evaluating runs, and comparing runs. It also includes ways to automatically download and process datasets.

An example of what is possible with this module:

>>> from pybool_ir.experiments.retrieval import RetrievalExperiment
>>> from pybool_ir.experiments.collections import load_collection
>>> from pybool_ir.index.pubmed import PubmedIndexer
>>> from ir_measures import *
>>> import ir_measures
>>>
>>> # Automatically downloads, then loads this collection.
>>> collection = load_collection("ielab/sysrev-seed-collection")
>>> # Point the experiment to your index, your collection.
>>> with RetrievalExperiment(PubmedIndexer(index_path="pubmed"),
...                                        collection=collection) as experiment:
...     # Get the run of the experiment.
...     # This automatically executes the queries.
...     run = experiment.run
>>> # Evaluate the run using ir_measures.
>>> ir_measures.calc_aggregate([SetP, SetR, SetF], collection.qrels, run)

Modules

pybool_ir.experiments.collections	Classes and methods for loading collections.
pybool_ir.experiments.decompose	Classes and methods for running experiments that involve execution of each atomic node in a query.
pybool_ir.experiments.retrieval	Classes and methods for running retrieval experiments.
pybool_ir.experiments.smooth	Classes and methods for the paper "Smooth Operators for Effective Systematic Review Queries".