Jobeffekter.dk is an internet-based knowledge bank where users have access to studies of labour market programmes. Independent researchers decide which research papers are added to the database, each paper is awarded a quality stamp, and a star rating indicates the aggregated effect of the paper.
The purpose of the database is to provide easy access to high-quality research on the overall effects of labour market programmes. The database is maintained independently by the Ministry of Employment.
The structure of the database is grounded in three basic principles, which are aimed at ensuring independence, transparency, and that the contents of the database are based on high-quality research:
Decisions concerning the content and quality assessment of the database are made solely by independent researchers.
The algorithm used to compare and calculate overall findings for the effects of labour-market programmes are determined solely by independent researchers.
The methodology applied in the quality assessment and the algorithms used to calculate the overall effects are transparent and made available to all users of the database.
Procedures for managing research papers
As mentioned above, decisions regarding which research papers are added to the database are taken solely by independent researchers. The guiding principle used to determine whether studies are added to the database is the use of quality assessment reviews. The researcher who carries out the review reaches a decision as to whether a given paper is of sufficient quality to be included in the review and added to the database.
Once papers are added to the database, the researcher who has carried out the quality assessment review has the responsibility of keeping the database updated with respect to the topic of the review in question. Other researchers may recommend studies to be added to the database, but it is the responsible researcher who makes the final decision as to whether a given study is ultimately included. For each paper, or for each result/conclusion in a given paper, the researcher adds the following information to the database:
- Target group (long-term unemployed, with education, without education)
- Intervention (The labour market policy which is evaluated in the paper)
- The methodology of the study (controlled experiment, effect-study of panel data or quantitative evaluation)
- Research year
- Country of research
- Publisher of the paper
- Gender of the target group
- Age of the target group
- The effect (positive, negative or no effect)
- Number of observations in the study
The results of papers will vary from study to study, but will always pertain to the labour market, e.g. jobs, education, employment rates, cost-benefit analysis etc.
The researcher has the task of carrying out a quality assessment of the given study. There are four different quality stamps:
- Excellent analysis – No remarks.
- Potential problems with the analysis, but not of major concern with respect to its results.
- Problems with the analysis which are expected to affect the study's findings.
- Quantitative evaluations are not given any quality stamps.
The guidelines for which quality stamps are awarded to each paper are drawn up in collaboration with independent researchers and are based upon a range of considerations:
The quality of the data used in the paper, the robustness of the findings, the number of investigations carried out, the assessment of control variables etc. For each paper in the database, it is possible to view the name of the researcher who added the paper. Researchers also have the possibility to comment on quality stamps awarded by other researchers.
In order to calculate the overall effect of a given labour market programme on a particular target group, thereby making results in the database easily accessible to the layman, it is necessary to aggregate the results for each study. The overall effect can be assigned one of four values:
- Positive effect
- Negative effect
- No effect
- Ambiguous effect or insufficient knowledge
Aggregation means that every parameter in the studies included in the database is assigned a certain weight. Weighting is determined by independent researchers. Weighting takes into account: The methodology of the study, the country of research, the quality of the study as rated by the researcher and whether or not the study is peer-reviewed.
Furthermore, the quantity and quality of the evidence (the number of quality-assessed) which forms the basis of the calculation of the aggregated effect is indicated by a star rating:
- Unambiguous evidence, strong results (***)
- Moderate evidence (**)
- Indication (*)
- No evidence (No stars)
A given study can be assessed as being based on strong evidence, moderate evidence or indicate that a particular labour market programme has a positive, negative or no effect. If the amount of data in the database is too limited (“No evidence”) to enable calculation of the effect of a given programme, it will be classified as “uncertain”.
Researchers update the database on a regular basis to include new research. It is expected to be a useful tool for a wide range of people: policy makers, municipalities, researchers etc.
Jobeffekter.dk is accessible to the general public from the link below: