Open PhD position at TU Delft, The Netherlands

This is a PhD position open at the Computer Science Department.

However, applicants with Statistics background are specially welcomed.

Full information:


Evaluating Information Retrieval and Recommendation systems, as well as other dataset-trained systems, has proved to be challenging. There are many factors known to affect the results we obtain from an experiment, such as the dataset, metric, data partitions, network initialization, statistics, etc. Many other factors are typically decided upon only once, and researchers just follow the same methodologies in their everyday research: data collection, dataset size, metric design, user models, annotators, statistical analysis, interpretation of results, missing ground truth, etc.

Many of these factors are known to have a large impact on our evaluation experiments, even to the point of questioning the trustworthiness of the results and the conclusions drawn thereof. Research over the past two decades has come up with ways to minimize this impact and allow us to have a more robust understanding of the field. However, how to best deal with others is an open research problem. This PhD project will continue current research to explore ways to increase the reliability of experiments in Information Retrieval and related fields.

You will have the opportunity to address some of the following topics:
- Test collection design.
- Handling of missing judgments.
- Ranking of retrieval systems.
- Statistical analysis of evaluation results.
- Simulation of test collections
- Development of evaluation tools

Research output is expected in the form of papers to be published in leading venues like SIGIR, RecSys, CIKM, ECIR, WSDM, WWW, TOIS or IRJ, software packages and datasets, with a very clear contribution to Open Science and reproducible research.


The successful PhD candidate has an MSc in Statistics, Computer Science, or a closely related field. Solid foundation in one or more of the following disciplines is very much looked for: information retrieval, recommender systems, machine learning, statistical modeling, data analytics. We require programming skills in modern languages, preferably in R. We require fluently spoken and written English. In addition, we are looking for creativity, critical thinking, rigor, independence, and good communication skills.


- Fixed-term contract: 4 years.
- 38—40 hours per week
- €2325—€2972 per month

The PhD candidate will be part of the Evaluation Lab and work under the supervision of Assistant Professor Julián Urbano (


For information about this vacancy, you can contact Julián Urbano, email:, tel: +31 15 2786176.

For information about the selection procedure, please contact Karin Reijenga, HR officer, email:

All applications should include a PDF attachment with:
- a detailed CV (including list of publications if available).
- a statement of research interests.
- a list of courses taken and grades obtained.
- a copy or link to your Master's thesis.
- contact details of 2-3 references.

You can send your application material before March 21, 2020 to: vacancies-eemcs@tudelft, referring to vacancy number EWI2020-16.

Zurück zur Übersicht