Publications of the Ministry of Economic A�airs and Employment • Integration • 2022:27 Systematic Review of Active Labor Market Policies’ E�ects on Immigrants’ Employment Systematic Review of Active Labor Market Policies' Effects on Immigrants' Employment Sanni Kiviholma, Hannu Karhunen Ministry of Economic Affairs and Employment of Finland Helsinki 2022 Publications of the Ministry of Economic Affairs and Employment 2022:27 Ministry of Economic Affairs and Employment of Finland This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited. ISBN pdf: 978-952-327-638-3 ISSN pdf: 1797-3562 Layout: Government Administration Department, Publications Helsinki 2022 Finland Publication distribution Institutional Repository for the Government of Finland Valto julkaisut.valtioneuvosto.fi Publication sale Online bookstore of the Finnish Government vnjulkaisumyynti.fi https://julkaisut.valtioneuvosto.fi/ https://vnjulkaisumyynti.fi/ Description sheet 30 March 2022 Systematic Review of Active Labor Market Policies' Effects on Immigrants' Employment Publications of the Ministry of Economic Affairs and Employment 2022:27 Subject Integration Publisher Ministry of Economic Affairs and Employment of Finland Author(s) Sanni Kiviholma, University of Jyväskylä and Labour Institute for Economic Research LABORE, sanni.kiviholma@labore.fi (corresponding author) Hannu Karhunen, Labour Institute for Economic Research LABORE, hannu.karhunen@labore.fi Language English Pages 44 Abstract This systematic review describes the recent literature on integration programs and active labor market policies (ALMPs) that strive to enhance immigrants' employment. We searched several databases, resulting in a total of 2,514 records. After step-by-step screening, we had 44 studies that satisfied our predetermined inclusion criteria. We divided policy measures into four groups: language training, integration programs, monetary sanctions and general ALMPs. Although these policies often overlap, the studies focus on certain parts of the policies. The main finding of this review is that well executed integration measures can improve immigrants' labor market attachment, speed up the process of entering employment, and improve the quality of attained jobs. Still, thorough analyses of ALMPs' effects on immigrants are scarce, and the results are inconclusive. Labor market institutions, immigrant populations, and the history of immigration are different even among the Nordic countries, which is why the results are not directly applicable to the Finnish context and there should be more causal research on the topic from Finland. To achieve this, the Finnish government should start experimenting with ALMPs but also implement possible new policies in a way that enables convincing causal analysis on policy effects. Keywords integration, immigrants, individual integration, work, employment, employment policy ISBN PDF 978-952-327-638-3 ISSN PDF 1797-3562 URN address https://urn.fi/URN:ISBN:978-952-327-638-3 https://urn.fi/URN:ISBN:978-952-327-638-3 Kuvailulehti 30.3.2022 Systemaattinen katsaus aktiivisen työmarkkinapolitiikan vaikutuksiin maahanmuuttajien työllistymisessä Työ- ja elinkeinoministeriön julkaisuja 2022:27 Teema Kotouttaminen Julkaisija Työ- ja elinkeinoministeriö Tekijä/t Sanni Kiviholma, Jyväskylän yliopisto ja Työn ja talouden tutkimus LABORE, sanni.kiviholma@labore.fi (vastaava tekijä) Hannu Karhunen, Työn ja talouden tutkimus LABORE, hannu.karhunen@labore.fi Kieli englanti Sivumäärä 44 Tiivistelmä Tässä systemaattisessa katsauksessa kuvataan viimeaikaista kirjallisuutta, joka käsittelee maahanmuuttajien työllistymisen edistämiseen tarkoitettuja kotouttamisohjelmia ja aktiivista työmarkkinapolitiikkaa (ALMPs). Katsausta varten tehtiin hakuja useista tietokannoista, joista löytyi yhteensä 2 514 merkintää. Vaiheittaisen seulonnan jälkeen todettiin 44 tutkimuksen täyttävän ennalta määritellyt kriteerit katsaukseen sisällyttämiselle. Poliittiset toimet jaettiin neljään ryhmään: kielikoulutukseen, kotouttamisohjelmiin, tulonsiirtoihin ja yleiseen aktiiviseen työmarkkinapolitiikkaan. Vaikka nämä toimet ovat usein päällekkäisiä, tutkimukset keskittyvät niiden tiettyihin osiin. Katsauksen keskeisin havainto on se, että onnistuneesti toteutetuilla kotouttamistoimilla voidaan parantaa maahanmuuttajien kiinnittymistä työmarkkinoille, nopeuttaa heidän työllistymistään ja auttaa heitä saamaan parempia työpaikkoja. Perusteellisia analyyseja aktiivisen työmarkkinapolitiikan vaikutuksista maahanmuuttajiin on kuitenkin tehty vain vähän, eikä niiden tulosten perusteella voida tehdä varmoja päätelmiä. Työmarkkinalaitokset, maahanmuuttajaväestö ja maahanmuuton historia ovat eri Pohjoismaissakin erilaisia, ja siksi tulokset eivät ole suoraan sovellettavissa Suomen olosuhteisiin ja aiheesta tulisi tehdä enemmän kausaalitutkimusta Suomen oloista. Tämä edellyttää, että Suomen hallitus hyödyntää satunnaiskokeiluja aktiivisen työmarkkinapolitiikan suunnittelussa ja toimeenpanee uudet lait tavalla, joka mahdollistaa luotettavan syy-seuraus-analyysin politiikan vaikutuksista. Asiasanat kotouttaminen, maahanmuuttajat, kotoutuminen, työ, työllisyys, työvoimapolitiikka ISBN PDF 978-952-327-638-3 ISSN PDF 1797-3562 Julkaisun osoite https://urn.fi/URN:ISBN:978-952-327-638-3 https://urn.fi/URN:ISBN:978-952-327-638-3 Presentationsblad 30.3.2022 Systematisk studie av aktiva arbetsmarknadsåtgärders påverkan på sysselsättningsgraden hos invandrare Arbets- och näringsministeriets publikationer 2022:27 Tema Integration Utgivare Arbets- och näringsministeriet Författare Sanni Kiviholma, Jyväskylä universitet och Labour Institute for Economic Research LABORE, sanni.kiviholma@labore.fi (korresponderande författare) Hannu Karhunen, Labour Institute for Economic Research LABORE, hannu.karhunen@labore.fi Språk engelska Sidantal 44 Referat Denna systematiska studie beskriver den senaste litteraturen om integrationsprogram och aktiva arbetsmarknadsåtgärder vars syfte är att öka sysselsättningsgraden hos invandrare. Vi sökte i flera databaser och fann sammanlagt 2 514 poster. Efter en stegvis urvalsprocess valde vi ut 44 studier som uppfyllde våra på förhand fastställda urvalskriterier. Vi delade upp åtgärderna i fyra grupper: språkutbildning, integrationsprogram, penningöverföringar och allmänna aktiva arbetsmarknadsåtgärder. Medan dessa åtgärder ofta överlappar varandra fokuserar studien på specifika delar av åtgärderna. Studiens huvudsakliga resultat är att väl utförda integrationsåtgärder kan förbättra invandrares inträde på arbetsmarknaden, påskynda anställningsförfarandet och förbättra kvaliteten på de tjänster som erhålls. Noggranna analyser av de aktiva arbetsmarknadsåtgärdernas effekter på invandrare är dock fåtaliga och resultaten är ofullständiga. Arbetsmarknadsinstitutioner, invandrarpopulationer och invandringens historia skiljer sig även mellan länderna i Norden. Därför kan resultaten inte tillämpas direkt på Finland och det behövs mer kausal forskning på området från finländskt håll. För att uppnå detta bör den finska regeringen börja experimentera med aktiva arbetsmarknadsåtgärder men samtidigt genomföra möjliga nya åtgärder på ett sätt som möjliggör övertygande kausal analys om åtgärdernas verkan. Nyckelord integration, invandrare, integrationsfrämjande, arbete, sysselsättning, arbetskraftspolitik ISBN PDF 978-952-327-638-3 ISSN PDF 1797-3562 URN-adress https://urn.fi/URN:ISBN:978-952-327-638-3 https://urn.fi/URN:ISBN:978-952-327-638-3 Contents Abstract.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1 Introduction.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1 Search Strategy.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Inclusion Criteria.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Study Selection and Data Extraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Results.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1 Characteristics of the Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 Characteristics of Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3 Policy Effects.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.3.1 Studies Based on Experimental or Quasi-experimental Methods.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3.2 Studies Based on Matching Methods.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3.3 Synthesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.4 Comments on Internal and External Validity.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4 Conclusion.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 References.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Appendices.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 7 Publications of the Ministry of Economic Affairs and Employment 2022:27 A B S T R AC T This systematic review describes the recent literature on integration programs and active labor market policies (ALMPs) that strive to enhance immigrants’ employment. We searched several databases, resulting in a total of 2,514 records. After step-by-step screening, we had 44 studies that satisfied our predetermined inclusion criteria. The first criterion was that the studies should investigate countries that are highly comparable to Finland such as Germany, Sweden, and Denmark. Second, the search was limited to studies published after 2005, as causal estimation methods came into wider use in policy evaluation in the 2000s. This review focuses mainly on studies that offer experimental or quasi-experimental evidence on the effects of policies on immigrants’ employment, as these are regarded as the most reliable types of research methods. Third, we consider employment to be our main outcome of interest. Nevertheless, we also include studies that use income, labor market participation, and duration of unemployment as outcome variables. We divided policy measures into four groups: language training, integration programs, monetary sanctions, and general ALMPs. Although these policies often overlap, the studies focus on certain parts of the policies. Still, estimation results presented in this review often reflect one or a combination of several types of policies, which should be remembered when interpreting results on one specific policy. The presented studies also do not assess whether the estimation results are biased due to the displacement effect that may bias the results upwards. The main finding of this review is that well executed integration measures can improve immigrants’ labor market attachment, speed up the process of entering employment, and improve the quality of the attained jobs. Still, thorough analyses of ALMPs’ effects on immigrants are scarce, and the results are inconclusive. Labor market institutions, immigrant populations, and the history of immigration are different even among the Nordic countries, which is why the results are not directly applicable to the Finnish context and there should be more causal research on the topic from Finland. To achieve this, the Finnish government should start experimenting with ALMPs but also implement possible new policies in a way that enables convincing causal analysis on policy effects. 8 Publications of the Ministry of Economic Affairs and Employment 2022:27 1 Introduction The purpose of this study is to provide a systematic literature review of active labor market policies (ALMPs) and integration policies targeted at immigrants. It provides an update of an earlier literature review by Butschek et al. (2014) and can be used in planning policies and randomized controlled trials. Immigrants’ employment and successful integration into society is a central challenge for the public sector, especially in countries such as Finland, which faces problems stemming from an aging population and private sector labor shortages. Despite this, robust evidence on effective policies is relatively scarce and scattered. The extensive meta-analysis of ALMPs by Card et al. (2018) does not consider immigrants or other subgroups separately, but ALMPs are likely to impact different groups significantly differently. Sarvimäki and Hämäläinen (2016) showed that even small activation measures can affect immigrants’ income and transfers, if the policies are well planned. The only previous meta-analysis of ALMPs’ effects on immigrants’ employment, by Butschek et al. (2014), presents an analysis of 33 studies yielding 93 estimates. The review focuses on the effects of four types of ALMPs: training, job search assistance, wage subsidies, and public sector employment. It does not offer estimates on the effects of language training or other policies designed specifically for immigrants. The meta-analysis concludes that wage subsidies increase immigrants’ employment. Overall, their findings do not contradict the results of previous research on all unemployed immigrants. In this review, we narrow our search to papers released after 2005 because causal estimation methods came into wider use in policy evaluation in the 2000s. The majority of older research results are based on a matching approach that relies on control groups constructed from the available data. We have strict requirements for inclusion, and our scope is not as wide as in the previous analysis by Butschek et al. (2014). We focus mainly on studies that offer experimental or quasi-experimental evidence on the effects of policies on immigrants’ employment, as these are generally considered the most reliable research methods. Active labor market policy evaluations come mostly from the Nordic countries, which we consider most comparable to Finland. Defining public policies that target immigrant populations in different countries is not an easy task. Both unemployed workers and immigrants can be subject to ALMPs, some of which are targeted at all jobseekers, and others are aimed specifically at immigrants as a 9 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 part of integration policies. General ALMPs are typically divided into training, subsidized employment in the private or public sector, and job search assistance and sanctions (Butschek et al., 2014). Programs for immigrants often consist of a mix of language training and introduction programs. In this review, however, we aggregate policies for unemployed immigrants into four groups: general ALMPs, language training, integration programs, and monetary sanctions. The review is structured as follows. In the next chapter, we discuss the methods used for the systematic review. We show the selection process and discuss the selection criteria. In the third chapter, we discuss the effects of the policies and draw a synthesis from the results. In the final chapter, we discuss the policy implications. 10 Publications of the Ministry of Economic Affairs and Employment 2022:27 2 Methods 2.1 Search Strategy Our main sources for records were three electronic databases: Web of Science, Scopus, and EconLit. We complemented the search with articles from various sources that publish working papers. We focused on the most recent studies on the topic, limiting the search to studies from 2005 onwards. We searched for studies in English. Our primary search included words from four categories: research methods, immigration, policies, and employment. The search terms included words such as immigrant, migrant, non-native, labor market, active labor market policies, integration, and employment, as well as different combinations of these words and abbreviations related to them. The search strategy varied slightly depending on the database. The complete search phrases are presented in Appendix A. The first database search was done on October 27, 2021. We exported all search results to Mendeley and created local .bib files. We found 741 results from Web of Science, 839 from Scopus, and 254 from EconLit, a total of 1,834 studies. We ran an additional search with alternative terms on Web of Science on November 1, 2021. This time, we narrowed down the search to articles in economics, which resulted in 238 unique studies. The academic publishing process can take several years, but policy-relevant studies are often published as early as possible as working papers before being sent to journals for peer review. We conducted additional searches from known channels that publish working papers to ensure we also included the latest research in the review. We searched for working papers directly from the IFAU, IZA, CES, and NBER websites1. The search tools on these sites are usually not as flexible as those of databases, so we ran several searches using individual keywords. On November 5, 2021, we looked for working papers from IFAU using two different searches: “active labor market polic” and “immigra”. These searches resulted in 45 working papers, starting in 2005. Similarly, IZA’s database was searched using the search term “active labor market policy,” and the search resulted in eight publications since 2005. We 1 The Institute for Evaluation of Labour Market and Education Policy (IFAU) is a Swedish state-owned research institute. IZA Institute of Labor Economics is a nonprofit research institute based in Germany. Center for Economic Studies (CES) is part of the United States Census Bureau. National Bureau of Economic Research (NBER) is a private organization from the United States that facilitates and publishes economics research. 11 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 also conducted a search of CES working papers and found 120 results. On November 8, 2021, we searched NBER working papers, which resulted in 150 studies. Finally, we used backward and forward searches. A backward search was done by checking the citations of the most recent included studies. This led to 20 additional studies. Using Google Scholar, we did a forward search by screening the 99 papers that cited the meta-analysis of ALMP studies by Butschek et al. (2014). 2.2 Inclusion Criteria We aimed to find all studies on the impacts of ALMPs on immigrants’ employment. We used the PICOS method (population, intervention, comparison, outcomes, and study design) in choosing the articles to clarify the inclusion criteria (Connolly et al., 2018). The population of interest was immigrants who participate in integration schemes or ALMPs. Interventions included active labor market programs, benefits, and integration measures targeted at all unemployed workers or immigrants. We required that the chosen studies have a comparison group. Our focus was on causal employment outcomes, but we considered other outcomes, such as crime rate, exit rate from unemployment, income, and language skills. The impact of ALMPs has been studied using various econometric methods. We focused on studies that utilize randomized controlled trials (RCTs), regression discontinuity design (RDD), instrumental variable (IV) methods, and difference-in-differences (DID) settings because these are considered the most reliable by researchers. Many previous analyses also used propensity score matching for estimating causal effects. In propensity score matching, causal effects are estimated by forming a control group that is identical to the treatment group. This is usually not possible due to unobserved characteristics in the groups. Today, propensity score matching is not considered a reliable method of estimating causal effects. We include these studies but discuss the results separately in Part 3.3.2. We prioritized studies from countries that are the most comparable to Finland, such as the Nordics, but we do not exclude studies from other countries. Comparisons between countries are more difficult the more different their labor market institutions are. We considered both the long-term and short-term effects of ALMPs and integration programs. Some ALMPs have lock-in effects; for example, participation in a training program may prevent unemployed jobseekers from applying for jobs during the training. On the other hand, some forms of ALMP may have instant effects; for example, training in a firm leads to better contacts and employment. We discuss these issues in the context of this review in Part 3.4. 12 Publications of the Ministry of Economic Affairs and Employment 2022:27 2.3 Study Selection and Data Extraction We selected studies for further evaluation using the revtools package for R (Westgate, 2019). We removed duplicates, screened the abstracts from our search results, and chose the studies to be included. The results were exported to an Excel spreadsheet for a second round of evaluation and data extraction. We chose 67 records for full-text analysis. Finally, the screening process led to a set of 44 unique studies that we discuss in this literature review. The selection process is described in Figure 1. Figure1.  Research selection process Id en ti� ca tio n Sc re en in g Eli gib ilit y In clu de d Database search n = 2072 Working papers n = 323 Backward and forward searches n = 119 Records excluded n = 2383 Records excluded n = 64 Records after abstract screening n = 131 Full text articles assessed n = 67 Studies included in synthesis n = 44 Records in total n = 2514 Records excluded n = 23 13 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 3 Results 3.1 Characteristics of the Research Table 1 shows the frequency and share of different study designs, primary outcomes, the location, and whether the research was published. Seventy percent of the records have been published in peer-reviewed journals. Of these, 18 used experimental or quasi- experimental research designs. This research represents various ways of causal inference. Experimental and quasi- experimental methods are considered more prudent estimation methods than methods based on matching (see e.g. Huntington-Klein, 2022; Angrist and Pischke, 2009). Studies using propensity score matching (PSM), other matching methods, and duration models represent half of the studies in this literature review (48%), but they are discussed in less detail because the method is not considered reliable for causal estimation. Randomized controlled trials (RCT) are considered the most reliable methods for causal inference. Of the studies selected for the evaluation, only three were RCTs. We grouped studies using difference-in-differences, triple difference, timing of events, event studies, and two-way fixed effects with the control group to a Before–After category that represents 30% of studies. Other widely used methods for policy evaluation are regression discontinuity design (RDD) and instrumental variable methods (IV). We consider employment our main outcome of interest. The studies measured employment very differently. Employment can mean, for example, the rate at which an unemployed individual finds work, or it can mean total hours worked during a decade (as in Sarvimäki & Hämäläinen, 2016). Some studies measured whether an individual was employed at all during a certain time span. These can be very different estimates, and comparisons between studies should be conducted with caution. We also considered studies that have other outcomes, but 93% of the studies included employment as a primary outcome. In one study, the primary outcome is income. Other outcomes are labor market participation and search behavior. Another difficulty in interpreting the results from different studies relates to the timing of outcome measurement. ALMPs and integration policies likely have different short- and long-term effects, but in practice, the follow-up period is limited for practical reasons. Nevertheless, when analyzing results, it is important to report when the outcome is measured and if there are differences over time. 14 Publications of the Ministry of Economic Affairs and Employment 2022:27 We focused on studies from countries that are most comparable to Finland, including the Nordics and northern and central Europe. These countries usually have strong unemployment protection laws, and the responses to ALMPs could thus be similar in these countries. The chosen records include studies on ALMPs and integration from Finland, Estonia, Sweden, Norway, Denmark, Germany, the Netherlands, France, Italy, the United States, and South Korea. The studies are mostly from Europe, with only five being from elsewhere. Most of the studies on ALMPs are from Germany, but almost all of them (11 studies) use matching methods, which are considered the weakest methods. The second most numerous studies are from Sweden. The Swedish studies use a mix of study designs, and the country’s labor market institutions are probably the most comparable to Finland. The third-highest contributor is Denmark, where more reliable methods, such as RDD and event study, were mostly used. 15 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 Table 1.  Characteristics of the chosen research Type Frequency % Study Design Matching 21 48 % Before-After 13 30 % RDD 5 11 % RCT 4 9 % IV 1 2 % Total 44 100 % Primary Outcome Employment 41 93 % Income 1 2 % Other 2 5 % Total 44 100 % Published Published 31 70 % Working Paper 13 30 % Total 44 100 % Location Estonia 1 2 % Finland 1 2 % France 1 2 % Italy 1 2 % Netherlands 1 2 % South Korea 1 2 % Norway 4 9 % United States 4 9 % Denmark 8 18 % Sweden 9 20 % Germany 13 31 % Total 44 100 % Figure 2 further shows the studies by the year they were published and the type of research design. The earliest studies included are from 2005. The figure clearly shows how the first decade of our timeline consists of mostly matching studies. Overall, 52% 16 Publications of the Ministry of Economic Affairs and Employment 2022:27 of the studies used an experimental or quasi-experimental research design. There are, on average, three studies per year during the reference period, while no studies are from 2014. The latest study in this review is from 2022 (Arendt, 2022) and it was already available online at the time of the search. Figure2.  Selected studies by year published and research method 3.2 Characteristics of Policies Table 2 depicts the characteristics of the ALMPs and integration policies in the research separately for matching studies and studies using experimental or quasi-experimental methods. We consider the target group of the policy: whether it was a general policy, or a policy aimed specifically at immigrants. We also show the shares of different types of policies considered in the research and the decade of policy implementation. Papers using experimental and quasi-experimental methods mostly study policies for immigrants (70%) and, to a lesser extent, policies that are meant for all unemployed individuals. Overall, a little over half of the studies (55%) estimated the effects of policies intended for all unemployed individuals, but these report the effects of the program on immigrant populations separately. Fewer than half (45%) of the studies focused on programs targeted specifically at immigrants. It should be stressed that we might not know all the different policy goals that programs in different countries may originally have had, as we limit our analysis to selected studies that may focus on specific measurable goals such as employment. 1 2 2 1 2 2 2 1 4 3 2 1 1 1 2 3 3 2 1 1 3 1 1 2 0 1 2 3 4 5 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Experimental and quasi-experimental methods Matching methods 17 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 We divided the types of programs into four groups for simplicity: integration programs, language training, monetary sanctions, and ALMPs. The first group includes integration programs targeted at newly arrived immigrants (labor market training, language courses, and counseling). These programs are often a mix of traditional ALMPs and programs for immigrants specifically (16%). Of the experimental studies, they represent a fourth. The second group is language training (14%). Even though it is usually combined with other forms of ALMPs, these studies focus clearly on estimating the causal effects of language learning. Four experimental or quasi-experimental articles studied the effects of language training. The third group (27%) consists of studies that analyze the effects of monetary schemes used to increase or decrease different forms of benefits unemployed workers or immigrants receive. Ten experimental studies looked at the effects of monetary assistance. The final and the largest group includes traditional labor market policies, such as subsidized labor, labor market training, counseling, monitoring, and sanctions. These policies are targeted at all unemployed workers, with 43% of the studies analyzing the effects. Most of the matching studies (76%) look at the ALMPs’ effects on unemployed workers and report the results for immigrants separately. In the last part of the table, we show the timing of when the program came into effect, or when the effects were studied, by decade. The policies analyzed in the research are mostly from the 2000s. Fifty-two percent of experimental or quasi-experimental research studied policy changes that took place in 2000s, and 26% took place in 1990s. Only five experimental and quasi-experimental studies investigated policy changes from the 2010s. 18 Publications of the Ministry of Economic Affairs and Employment 2022:27 Table 2.  Characteristics of policies Type Experimental Matching Total Frequency % Frequency % Frequency % A) Policy target Immigrants 16 70 % 4 19 % 20 45 % General 7 30 % 17 81 % 24 55 % Total 23 100 % 21 100 % 44 100 % B) Policy type ALMP 3 13 % 16 76 % 19 43 % Integration program 6 26 % 1 5 % 7 16 % Language training 4 17 % 2 10 % 6 14 % Government transfers 10 43 % 2 10 % 12 27 % Total 23 100 % 21 100 % 44 100 % C) Timing of program 1990s 6 26 % 4 19 % 10 23 % 2000s 12 52 % 13 62 % 25 57 % 2010s 5 22 % 4 19 % 9 20 % Total 23 100 % 21 100 % 44 100 % 3.3 Policy Effects Many countries offer immigrants some type of integration program or other support that strives to improve their employment prospects in a new country. Next, we present results from the studies that are based on experimental or quasi-experimental methods, which we consider to be the most reliable approaches to studying how different policies affect employment. Table 3 presents the details of these studies. We also comment on research results that are based on matching approaches in Section 3.3.2. 19 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 3.3.1 Studies Based on Experimental or Quasi-experimental Methods Immigrant Integration Programs Integration programs are a mix of language training, host-country-specific training on culture and customs, and general ALMPs that are also offered to other unemployed individuals. In this chapter, the studies evaluated integration programs that consist of several different components, and it may not be possible to attribute the effects to a specific part of the program. The main finding from this literature is that well executed integration measures can improve immigrants’ labor market attachment and speed up entrance to employment. The studies on integration programs are from the Nordic countries: Denmark, Norway, Sweden, and Finland. Arendt (2022) studies the effects of Denmark’s work-first policy, which aims to speed up refugees’ entry into the labor market. The policy was introduced in 2016 after the surge of refugees to Europe, and it requires refugees to both search for jobs as well as attend on- the-job training. Work-first policies emphasize a quick labor market entry of immigrants instead of other forms of integration, such as language training. Arendt (2022) used a quasi-experimental before–after design that compares the employment paths of refugees who arrived before and after the introduction of the policy. The group that arrived before the introduction of the program was not affected by it and thus formed the control group. Their estimates describe an intention-to-treat effect: the treatment group arrived after the reform, but it is not known whether they participated in the program. The main assumption is that in the absence of the policy, the employment of the control group and treatment group would have developed similarly. The results show that the policy had no effect on women’s employment, but men found work faster: within 11 months of arrival, men’s employment improved by 33%, and after 13 months by 59%. The significant positive effect of the program is only seen after 11 months, but the effect wears down at 15 months. The differences between men’s and women’s outcomes may be due to several reasons. For instance, men might participate more in job training. Participation in job training can partially crowd out other forms of integration, such as language training. Even though the study found positive effects on men’s employment during the first year, there was no effect on receiving welfare benefits. This may suggest that employment is part- time or short-term. Arendt (2022) mentions three possible biases in their study. First, the earlier cohort and the later cohort may be different in unobservable characteristics, which can cause a downward bias in the estimates. Second, employers may have increased their demand for employing refugees in 2016, independently of the work-first policy, causing a downward bias as well. Third, the later cohort had to wait longer for an asylum decision and had more time to develop host country-specific skills, such as the host country language. 20 Publications of the Ministry of Economic Affairs and Employment 2022:27 Ugreninov and Turner (2021) studied the effect of a Norwegian introduction program on female immigrants’ employment. The program offered immigrants language training, education, and on-the-job training to improve their employment and earning prospects. They studied employment effects using a difference-in-differences framework in which the treatment group was immigrants entitled to the introduction program after the reform, and the control group consisted of Norwegian-partner immigrants who were not eligible for the program at any time. Participation in the program was mandatory for refugees, while optional for family-reunited immigrants. These groups were compared before and after the reform to estimate the causal effect of the integration program. Their results show significant improvement (8–9 percentage points) in their employment probability after four to six years. Yet, the integration program did not affect their earnings, which suggests that the quality of the employment was not improved by the program. Dahlberg et al. (2020) studied the employment effects of an integration program provided by the city of Gothenburg in Sweden, in which 140 potential participants were randomly assigned to treatment and control groups, thus enabling relatively robust evaluation of the program effects. In addition to the city and the local public employment office, local firms participated in planning the program. Only recent immigrants with low levels of education were eligible to participate. The duration of the integration program was one year, and after the initial meeting, the program included three phases. First, the participants took part in intensive language training for three months. Next, they participated in supervised work practice for at least six months. Finally, they received job search assistance from local employment offices. The control group received the baseline services that public employment offices offered immigrants. The main difference between the control and treatment groups was the intensity at which the services were provided. The treatment group received training and other assistance throughout the year. Participants engaged in the program full time, and the aim was to learn a profession and gain full-time employment within two years of starting the program. Dahlberg et al. (2020) found a significant, 15 percentage point increase in employment compared to the control group in each month of the second year after the program started. The effect was significant, but the sample size was very small. The estimate describes an intention- to-treat effect: out of the 70 assigned to the program, 44 eventually participated. It may not be possible to reproduce these results on a larger scale due to costs associated with intensive training and job search assistance. The authors argue that displacement effects due to the program are unlikely because the program offered training in occupations with labor shortages and its target group is small despite it being implemented nationwide. Also, the possible displacement of unemployed workers closer to the labor market can be deemed acceptable. A cost–benefit analysis of the program is not included in this study. Sarvimäki and Hämäläinen (2016) published a seminal Finnish paper studying the effects of integration plans on immigrants’ income and employment. They leverage a 1999 21 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 policy that introduced integration plans for immigrants arriving in Finland. Immigrants who had arrived before a cutoff date were not eligible for the plan, so they utilized this discontinuity in estimating the effects. The reform did not increase the budget for ALMPs but rather restructured the services. After the reform, immigrants were able to obtain integration plans in the first three years of their stay. The plans included language courses and other training aimed specifically at immigrants. They found that the immigrants who received integration plans had 47% higher cumulative earnings during the 2000s than they would have without the plans. Their results do not show an impact on employment but do find a decrease in welfare benefits of 13%. Andersson and Nekby (2012) evaluated the labor market effects of a coaching program for new immigrants in Sweden. The participants in the program were randomized in nine municipalities, which allows for causal interpretation. The treatment group received intensive coaching and counseling from their assigned case workers at a local public employment office. New immigrants who were not assigned to the treatment group formed the control group, and they received traditional introduction services. The caseworkers who were assigned to immigrants in the treatment group had considerably fewer cases per month than the average case worker. The ALMP measures offered to immigrants in the treatment group were the same as those offered to the control group, but the reduced caseload allowed for more intensive counseling. The results of the study show that the treatment group was 3.2 percentage points more likely to be employed one year after registration, and 6 percentage points more likely to be employed than the control group 22 to 30 months after registration to the employment office. These results are promising but should be considered with caution, as the assignment of participants may not have been completely random despite the written instructions for program assignment. There could have also been spillover effects from the services offered to the treatment group to the control group, which would cause a downward bias. The authors’ cost–benefit analysis suggests that the costs of the program are covered within two to three years after the start of the coaching. Åslund and Johansson (2011) studied a pilot program offered by Swedish public employment offices in which participants were assigned to a caseworker who helped them find employment in several steps. First, the officer interviewed the client to map out the client’s skills and requests. Next, the caseworker came up with potential jobs for the client and analyzed whether they were appropriate. After the case officer checked with the employer that the tasks were suitable, the client started the workplace introduction. The caseworker followed up with the client after six months. The eventual aim of the introduction period was for the client to be hired. In the pilot, the caseworkers had 15 to 30 clients, when they typically dealt with ten times as many clients. While the program was originally developed for disabled workers as a 22 Publications of the Ministry of Economic Affairs and Employment 2022:27 supported employment program, the pilot program targeted unemployed immigrants who were at risk of long-term unemployment. For identification, the authors used a difference-in-differences method in which the unemployed immigrants in 20 participating municipalities were compared to immigrants in other municipalities. The identifying assumption is that the trend would have been the same in the participating municipalities and the control municipalities in the absence of treatment. The program participants’ hazard rate to employment increased by 12%, but it cannot necessarily be interpreted as a causal effect, as the trends in the treatment and control groups were not similar enough before the treatment. Each year of employment created is estimated to cost about 30,000 euros. Four of these six studies on integration polices showed a significant positive effect on employment. Åslund and Johansson (2011) also find a positive effect on employment, but their results should be treated with caution, as the results are not robust. While Sarvimäki and Hämäläinen (2016) do not find an effect on employment, their results show an increase in cumulative earnings and a decrease in the use of welfare benefits, which implies the success of the program. These studies are from Finland and countries most similar to Finland, and they suggest that integration programs do work to induce faster employment for immigrants. However, the programs themselves differ, and it is difficult to attribute the impact to specific parts of the program. Language Training Immigrants are often offered language courses as part of their integration into the host country.2 Language training can influence labor market outcomes through various channels, both directly and indirectly. The direct effects are the ability to perform tasks that require host country language knowledge. Language skills also indirectly signal potential employers of applicants’ human capital and commitment to integrating into the host country. Immigrants can be offered language training full-time or part-time, and during participation, courses often have lock-in effects – i.e., the participants are not available for employment during the course. Studies on language training show a positive effect on labor market integration, but the results for employment are mixed. The research on language training is from Germany, the United States, Denmark, and France. Lang (2021) analyzed the effects of a language program in Germany. The program included firm visits and internships, which are in themselves forms of active labor market policies. The 2 Åslund and Engdahl (2018) studied whether monetary incentives could increase language course achievement in Sweden. Their results showed no increase in participation in the municipalities that gave out bonuses for participants. 23 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 language course focused on language skills needed in the labor market, such as writing job applications. Participants in the program needed to have basic knowledge of German (A1 in the Common European Framework of Reference for Language), and they must be registered as jobseekers. To identify the causal effects of the program, the author used local treatment intensity as an instrumental variable. The local treatment intensity is the share of program participants out of all unemployed foreign-born individuals in the region. The instrumental variable method relies on the assumption that treatment intensity affects employment only through language course participation. This is a strong assumption, and recent studies (e.g., Blandhol et al. 2022) have challenged the use of instrumental variable methods in causal estimation. Lang’s (2021) results indicated that there is an initial lock-in period when unemployed workers attend the language course. In the first year after the start of the program, the probability of employment was significantly lower for participants (-15%), while it increased by 38% after two years. The results also suggest that the language course is more efficient if taken immediately after entering the labor force. Arendt et al. (2020) evaluated the effect of language training on refugees’ employment in Denmark. They leveraged a wider Danish reform from 1999 that focused on refugees’ language training. The main part of the reform was an increase in the hours of language training offered for refugees, as well as increased funding, and aimed to improve the quality of teaching. The reform also temporarily decreased benefits for refugees over 25 years old and refugees with children for 13 months. After the reform, refugees were placed in municipalities in proportion to the local population to allow for more dispersal, when previously the allocation was based on negotiations with the municipalities. The municipalities were then responsible for the implementation of integration programs, and the refugees’ benefits were tied into staying in the municipality of resettlement until the end of language training. Extended language training was mandatory for refugees arriving after a certain cutoff date, while those who received refugee status a day earlier received shorter training (which was also mandatory). This selection process of refugees for the program enabled the authors to use a regression discontinuity design in which the individuals granted refugee status just before and after the cutoff date are compared. As the refugees near the cutoff were not able to influence which group they ended up in, one can argue that, close to the cutoff, the participants were randomly assigned to the language course. It should be noted that the authors studied the reform as a whole and were not able to attribute the causal effects to specific components of the program. The findings of Arendt et al. (2020) show that the reform that emphasized language training for refugees had a positive effect on refugees’ earnings (34%) and employment (23%) 18 years after the reform. Their cost-benefit analysis shows that the costs of longer language training are covered in five years. 24 Publications of the Ministry of Economic Affairs and Employment 2022:27 Heller and Mumma (2020) evaluated the effect of language training on immigrants’ income in the United States. The participants in the language course were selected by a lottery from all applicants. They estimate that participation in the program increased the participants’ earnings (2–10 years after the program) 45–56% and that the additional tax revenue covered the cost of the program over time and generated a 6% annual return for taxpayers. Lochmann et al. (2019) studied the employment effects of language training in France. Immigrants who scored below a certain threshold on an initial language test were more likely to be assigned to a language course. This enables the use of a regression discontinuity design to study the effects of the language course. The method relied on the assumption that the individuals just above and just below the cutoff are similar, so that the assignment to treatment and control was essentially random. Immigrants were assigned a different number of hours of language training (0 to 400 hours). The results from Lochmann et al. (2019) showed that the language course had a positive effect on immigrants’ labor force participation, but not on their employment, at least in the short term. An increase of 100 hours of language training increased the probability of labor force participation by 14.5 to 26.6percentage points. The positive impact on labor force participation was stronger for individuals with higher education levels. They did not find differential effects by gender and age. The type of migrant, whether they migrated for labor, family, or refuge, also did not matter. Lochmann et al. (2019) discussed different channels through which local language skills can impact labor market participation. Improvement in language skills is a direct mechanism. A diploma from a language course may signal an individual’s language skills to potential employers, but their results did not support this hypothesis. They also considered the possibility that language courses and improved language skills helped immigrants create social networks but did not find evidence to support this, either. Language courses may also induce behavioral effects. The authors found evidence of negative behavioral effects through self-reported language skills and a lack of interest in the host country. The last channel is information: during the language course, the immigrants can learn new information about the host country from other participants and the teacher. The authors found evidence that the course indeed had positive effects in this way. These studies show that the improvement of language skills can improve labor market outcomes in the long term. Two of the studies found significant positive effects on employment after two and 18 years. Heller and Mumma (2020) found a positive effect on earnings. In the short-term, language courses seem to have lock-in effects, and the benefits may arise long after the language course ends, as the individuals’ language skills 25 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 keep improving. It is still important to note that four studies are not sufficient to make definite conclusions about the effects of language courses. Monetary Sanctions Decreasing benefits can be used to incentivize unemployed workers to find jobs. Some schemes may offer unemployed parents free childcare to allot time for job searches or work. The studies in this section show mixed results regarding the employment effects of benefit reductions. On the one hand, losing benefits seems to result in faster employment, but on the other hand, it can lead to dropping out of the labor force for some individuals. There is also evidence of negative well-being effects for the individual and their family members. In 2002, Denmark cut down welfare benefits for refugees in their Welfare Reform. The reform replaced social assistance with a lower “Start Aid.” Andersen, Dustmann, and Landersø (2019) studied the employment effects of benefit reduction. They leveraged the discontinuity around the time the new law was enacted and compared refugees who were granted asylum just before and after the cutoff. Refugees in both groups had arrived before the law was planned and enacted, so they were not able to affect their assignment. The authors find that the reduction had short-term positive employment effects: the effect was positive and significant one year after residency (89% compared to pre-reform mean) and two years after (37%). However, the reform decreased the net tax income of refugees by 40% over five years and caused a relatively large labor force exit of female refugees. LoPalo (2019) studied the effects of the level of cash assistance on refugees’ labor market outcomes, leveraging variation across U.S. states in levels of assistance for refugees in a difference-in-differences estimation. The results show that an increase of $100 in the maximum benefits is associated with a 5% to 8% raise in wages. The effects were strongest for the most highly educated individuals. The types of jobs refugees worked in were better, but the probability of employment did not change with the benefit level. East (2018) studied the employment effects of a change in food stamp eligibility in the United States. The Supplemental Nutrition Assistance Program (SNAP)is a federal, means- tested welfare benefit in the United States. In 1996, immigrants were excluded from the program, but soon after, individual states started to repeal the laws and give immigrants back their eligibility. East (2018) leveraged this variation between states and years to estimate the labor supply effects of gaining access to welfare benefits. They used two different specifications: double difference and triple difference. The former used state-year variation and estimates the labor supply decision. The latter compared immigrants eligible for food stamps to natives and uses state-year fixed effects. The fixed effects removed variations that affected both natives and immigrants at the same time in the same states. 26 Publications of the Ministry of Economic Affairs and Employment 2022:27 While natives may not be a good control group for immigrants, both specifications yield similar results. They found that, on average, single women were 6% less likely to work because of the benefit change. Men, however, did not change their labor supply on the extensive margin but did cut down their work hours by about 5%. Knoef and van Ours (2016) studied a policy pilot in the Netherlands, in which single mothers relying on welfare were able to work without losing their benefits. Before the policy change, benefits were cut down one-to-one when earning income. The policy targeted both native and immigrant single mothers. Using a triple difference set-up, Knoef and van Ours (2016) found that the intervention increased immigrant mothers’ employment with no change in native single mothers’ labor supply. The pilot had no effect on leaving welfare but did cut down welfare expenditures. The authors noted that the Netherlands has high levels of welfare benefits and, as a result, low incentives for work. A second part of the program combined earnings disregard with direct job creation where municipalities create jobs for the municipality or regular employers and connect jobseekers directly to them. Job creation had a positive effect on immigrant single mothers’ employment (18.7 percentage points). Drange and Telle (2015) studied the effect of free childcare on children’s educational outcomes and parental employment by leveraging an intervention that provided free childcare in some districts of Oslo, Norway. They used a difference-in-differences framework to study the causal effects. Free childcare increased participation rates and improved children’s educational outcomes. However, they did not find effects on parents’ employment. Formal childcare seems to be an important factor in reducing the gap in learning outcomes between native and immigrant children, but as an ALMP, at least in the context of this study, it is not efficient. In Sweden, Wikström et al. (2015) found that childcare reforms in 2001 and 2002 did not have an impact on immigrant mothers’ labor force participation or earnings, even though they had positive effects on native mothers. The reform aimed to increase formal childcare participation by reducing costs. Wikström et al. (2015) used a difference-in-differences approach comparing families before and after the reform. Mothers with children aged two to five are in the treated group, whereas mothers with 7 to 10-year-olds form the control group. The zero result is attributed to refugee immigrants’ other difficulties in the Swedish labor market. The problem may not be the lack of childcare, but rather other obstacles in finding employment. Vikman (2013) compared immigrant mothers who had access to paid parental leave when they arrived in Sweden to immigrant mothers who did not have access in order to study the effects on immigrant mothers’ labor force participation. The results showed that paid parental leave delayed women’s labor market entry. They use a difference-in-differences 27 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 setup that relies on two assumptions. First, only the age of the children affects labor force participation, and second, the effect is the same for immigrant mothers who migrated with children and immigrant mothers who had their children in the host country. In the treatment group were mothers whose youngest child was between the ages of two and six, while the mothers with children aged seven to 15 formed the control group. They also used a secondary control group, which accounted for differences in mothers’ labor market participation due to the age of the children. The second control group consisted of mothers who migrated to Sweden earlier and had children there. The results showed that mothers who were eligible for parental leave benefits were 7.7 percentage points less likely to be in the labor force two years after arriving. After seven years, there was no observable difference, which suggests that the benefits delayed the entrance. Vikman (2013) also showed that the employment rate of mothers was 3 percentage points lower after two to six years. Hardoy and Schøne (2010) studied the effect of a Norwegian benefit scheme on labor market participation for mothers who do not use publicly subsidized daycare. The reform introduced a cash benefit for childcare at home. The reform had negative effects especially on the non-Western immigrant mothers’ labor market participation. The authors use a triple difference estimation method in which they compare mothers eligible for the benefit to mothers with same-aged children in a different time period and who were not eligible. To account for macroeconomic shocks, they add a third difference, using mothers during the new legislation who had older children and thus were not eligible for the benefit. Their results show a 7 percentage point drop in the labor market participation of eligible parents. Rosholm and Vejlin (2010) studied the same Danish reform as Andersen et al. (2019), which reduced the public income transfers for refugees by 35%. The authors used a regression discontinuity design leveraging the discontinuity between refugees who received the lower benefits and those who were still eligible for higher benefits. The authors estimated that the reduction increased the likelihood of employment after two years by 140% but also increased the likelihood of exiting the labor force at the beginning of integration. The authors point out that benefit reduction does not automatically lead to faster employment, and labor market exits should be considered when planning welfare reforms. Kaestner and Kaushal (2005) leveraged a reform that excluded immigrants from welfare benefits to study the employment effects it had on low-educated, unmarried foreign- born women in the United States. The reform affected both earlier immigrants and immigrants arriving after the reform, but the change was more drastic for new immigrants. The authors compared immigrants’ employment before and after the reform using the difference-in-differences estimation method. Their results show mixed results, depending 28 Publications of the Ministry of Economic Affairs and Employment 2022:27 on the specification. However, they find that the most recent immigrants, who had arrived within five years, were the most affected by these policies: their employment increased by up to 4.1 percentage points (significant only at the 10% level). The five studies that analyzed the effects of reducing benefits for immigrants or excluding them from benefits showed mixed results on the effects. On one hand, the effects seem to be zero or slightly positive, but on the other, two studies show negative employment effects. Andersen et al. (2019) found that while the employment effects were positive, there were other adverse effects, such as a reduction in income and an exit from the labor market for women. The two studies that investigated free or cheaper childcare did not find employment effects. Paid parental leave showed positive effects on employment, and the possibility of working without losing benefits increased employment. General Active Labor Market Policies ALMPs often include job search assistance, training, monitoring job searches, and sanctions. Training can be either off-the-job or on-the-job and can include language courses and CV workshops. Job search assistance is usually offered by an assigned case worker at the employment office, and its aim is to connect unemployed workers to open positions that fit the candidate. Public employment offices’ periodical interviews with unemployed workers are a form of both monitoring and job search assistance. These studies analyzed the employment effects of job search assistance and sanctions. We also included temporary agency work in this category. The research in this section showed that those with the weakest attachment to the labor market can benefit the most from participating in active labor market programs. Battisti, Giesing, and Laurentsyeva (2019) conducted a randomized controlled trial in Germany to study the effect of job search assistance on refugees’ employment. Job search assistance was provided by an NGO that sent refugees’ CVs to suitable open positions. The effects were estimated using two surveys six and 12 months after the initial counseling sessions. Battisti et al. (2019) did not find significant effects on employment in the first six months, but after a year, there was a positive effect (31%) that is significant only at the 10% level. When they looked at the heterogeneous effects of the trial, job search assistance seemed to be most efficient for disadvantaged refugees: the ones with the lowest education levels and those who are still uncertain of their refugee status. The sample size in this study was small; the number of participants surveyed was 298 in the first round and 195 in the second round. Temporary agency work is not part of the public sector ALMP selection, but Jahn and Rosholm (2013) showed that it can help mitigate the information asymmetries related to hiring immigrants. They considered both first- and second-generation unemployed 29 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 immigrants in Denmark. Unemployed workers who took up temporary agency work at least once during their unemployment were considered treated. A random sample of all other unemployed immigrants form the control group. They used a timing-of-events set-up to estimate the connection between temporary agency work and employment during participation in temporary agency work and after. The number of past temporary agency work experiences was associated with the probability of falling back to unemployment for both men and women, suggesting that temporary agency work may stigmatize immigrant workers. Svarer (2011) studied the effect of sanctions on unemployment length for both natives and immigrants in Denmark. Receiving unemployment benefits requires, for example, that the unemployed look for jobs, participate in active labor market programs and accept jobs if offered. If they do not comply, public employment offices may impose sanctions. Sanctions can be a loss of unemployment insurance benefits for up to three weeks, or until the unemployed worker has worked for a certain number of hours in a given time period. The author found that sanctions significantly increased job-finding rates, and immigrants reacted to sanctions more strongly than natives. The exit rate to employment rate increases by over 100%. Unemployed workers also responded to less severe sanctions. The authors used a timing-of-events model to estimate the effects. According to these three studies, job search assistance and sanctions had positive effects on employment. Temporary agency work did not affect employment. Still, results from timing-of-events models should be interpreted with caution as they lack clear exogenous variation for causal analysis. The matching studies presented in the next section further discuss the policy effects of general ALMPs for immigrants. 30 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Table 3.  Selected studies (experimental or quasi-experimental) Study, * if published Country Policy Policy target group Method Effect on Employment 1 Arendt (2022) * Denmark Work-first policy: active job search and on-the-job training Immigrants Before-after Significant pos. 33-59 % effect for men one year since arrival. No effect for females. 2 Lang (2021) * Germany Language training Immigrants IV Significant neg. 15 % effect year after training, pos. 38 % effect 2 years after. 3 Ugreninov & Turner (2021) * Norway Integration program Immigrants DID Significant 8-9 %-point decrease in employment gap 4-6 years after arrival. 4 Arendt et al. (2020) Denmark Language training (inc. civic orientation, incentives, restrictions) Immigrants RDD Significant pos. 23 % effect (only at 10 % level) on employment 18 years after arrival/reform. 5 Heller & Slungaard Mumma (2020) United States Language training (ESOL program) Immigrants RCT Significant 45-56 percent increase in reported annual earnings (pooled earnings 2-10 years after) 6 Dahlberg et al. (2020) Sweden Integration program (inc. language training, search assistant and work practise) Immigrants RCT Significant 15 %-point increase in employment compared to the control group 7 Andersen et al. (2019) Denmark Reform that lowered the level of welfare benefits Immigrants RDD Significant pos. 37 % employment effect two years after for males, declining to 13 % in 3-5 years. Other adverse effects. 8 Battisti et al. (2019) * Germany Job search assistance (enhanced job-matching) Immigrants RCT No effect 6 months after, significant 31 % effect year after (only at 10 % level) 9 Lochmann et al. (2019) * France Language training Immigrants RDD No significant employment effect in the short term, significant effect on labour force participation. 10 LoPalo (2019) * United States Monetary assistance Immigrants DID No effect on employment; 5-8 percent increase in wages (1-4 years) 11 East (2018) * United States Food stamp access Immigrants triple DID Significant negative effect for married women (5 %) and single women (7 %) 31 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Study, * if published Country Policy Policy target group Method Effect on Employment 12 Knoef & van Ours (2016) * Netherlands Monetary assistant for single mothers (earnings disregard & direct job creation) Universal triple DID Significant 19 %-point increase in part-time work for immigrant single mothers compared to control group (natives). 13 Sarvimäki & Hämäläinen (2016) * Finland Integration plans (individualized sequences of ALMP) Immigrants RDD No effect on employment, but significant 47 % increase in cumulative earnings over decade among compliers. 14 Drange & Telle (2015) * Norway Free childcare for children aged 4 and 5 universal DID No effect on parental employment 15 Wikström et al. (2015) * Sweden Childcare reform (three main components) universal DID No effect on immigrant mothers 16 Vikman (2013) Sweden Access to paid parental leave Universal DID Significant 3 %-points effect employment 2-6 years after for immigrant mothers 17 Andersson Joona & Nekby (2012) * Sweden Intensive coaching and counselling (by PES) immigrants RCT Significant 3 %-points effect one year and 6 %-points effect 22-30 months after than control group. 18 Jahn & Rosholm (2013) * Denmark Temporary agency employment universal timing-of- events No effect on employment 19 Svarer (2011) * Denmark Sanctions universal timing-of- events Exit-rate from unemployment doubles following a sanction 20 Åslund & Johansson (2011) * Sweden Pilot program offered by PES (caseworker) immigrants DID Significant pos. 12 % effect on employment hazard, but results are not robust 21 Hardoy & Schøne (2010) * Norway Increase in price of day care universal triple DID Significant 7 %-point decrease in labor market participation 22 Rosholm & Vejlin (2010) * Denmark Cuts to "start-help" income immigrants RDD Transition rate to employment after 25 months rose by 140 % 23 Kaestner & Kaushal (2005) * United States Benefit cut immigrants DID No effect on employment 32 Publications of the Ministry of Economic Affairs and Employment 2022:27 3.3.2 Studies Based on Matching Methods The majority of the studies presented in Table 3 utilized exogenous variation to estimate the causal effect of different policies on immigrants’ employment. In practice, experimental or quasi-experimental approaches are rarely possible, as new policies are often implemented without an inbuilt research design (such as staggered rollout of policies). To overcome this problem, researchers strive to create as similar a control group as possible using data on those individuals who are not a target of the new program. The causal interpretation of these so-called matching results requires that, conditional on observed characteristics, the control group is similar to the treated group. Understandably, there may be many unobserved factors, such as inner motivation or unobserved skills, that can ruin this kind of reasoning. Nevertheless, we show the details of these studies in Appendix Table B1. To summarize, the matching results on employment are somewhat mixed, but do show a pattern. Most studies that originate especially from Germany and Denmark do suggest that traditional ALMPs or other policies that ease the transition to work are related to better employment one to two years after. Language training or subsidized vocational training is often part of the offered ALMPs. While these results are only descriptive, they point in a similar direction to the more prudent studies presented above. 3.3.3 Synthesis In this systematic review, we reviewed 44 studies that analyzed the effects of different policies that aim to improve immigrants’ employment. Out of these studies, 23 utilized experimental or quasi-experimental research methods, which we see as the most trustworthy. Six studies analyzed the employment effects of integration programs for immigrants in Sweden, Denmark, Norway, and Finland. These studies found positive effects on employment in the extensive margin (e.g., Arendt, 2022). Language training is often a part of integration programs, but it is surveyed separately in four studies. Two studies found positive effects on immigrants’ employment after two and 18 years. Butschek et al. (2014) found that, out of the ALMPs considered in their meta- analysis, only wage subsidies improve immigrants’ labor market outcomes. However, they did not study the effects of language training. This review showed that language training can work for immigrants with the lowest language skill levels and possibly the weakest labor market attachment. Language course attendance has lock-in effects, and the positive results from gaining language skills may be seen only in the long term. Ten studies looked at the effects of government transfers and monetary incentives on immigrants’ labor market outcomes. Lowering benefits may incentivize unemployed workers to take on jobs more quickly, but they might be only short-term or part-time work, 33 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 and the job may not correspond to the immigrant’s skill set. The results from these studies show mixed results on the employment effects of changes in government transfers. It is challenging to evaluate which policies would work in the Finnish context because the institutions and policies differ among countries. Integration programs seem to be an important part of immigrants’ integration, even though the contents of the programs vary by country. Immigrants are not a homogeneous group, and the studies often focus on only subgroups such as refugee men or immigrant mothers. 3.4 Comments on Internal and External Validity There are several concerns that could hinder the causal interpretations of the results presented in this review. Understandably, one cannot easily generalize research results from a different country to Finland if the country has, for example, different institutions and norms. Moreover, different estimation results often reflect one or a combination of several types of policies that are targeted at different immigrant groups, such as immigrant females or unemployed immigrants. Thus, the concerns about the results’ external validity should be noted when interpreting the results. Only one study in this review evaluates how immigrant policies affect immigrants’ employment in Finland. Researchers strive to improve the internal validity of research results by using advanced statistical methods that are based on experimental or quasi-experimental approaches. While these approaches are highly relevant for prudent causal analysis, there are also practical limitations that should be remembered. First, it is not clear when employment effects should be measured after the policy intervention because some ALMPs tend to have lock-in effects that decrease employment in the short term. Due to practical limitations, long follow-up periods after the ALMPs are not always possible. Second, there is evidence that individuals might anticipate policy changes, which could bias the interpretation of policy effectiveness (e.g., Filges and Hansen, 2017). This underlines the fact that it is not necessarily clear what part or mix of ALMPs causes the observed change in employment. Third, and perhaps most importantly, studies are rarely able to assess whether the estimation results are biased due to the displacement effect. If ALMPs increase immigrants’ employment at the expense of other unemployed individuals, then the true employment effect is smaller than what the estimation results indicate (see Crépon et al. 2013). Only one study of this review considers the possibility of the displacement effect (Dahlberg et al. 2020), arguing, that displacement effects are unlikely because the offered training is related to occupations with labor shortages. Nevertheless, the assumption of nonexistence of displacement effects is not clear in all reviewed studies. The displacement effects may also be larger if policies are implemented on a larger scale. 34 Publications of the Ministry of Economic Affairs and Employment 2022:27 4 Conclusion Integration of immigrants, especially into the labor market, continues to be a topic of discussion in Finland. This systematic review discusses recent literature on the effects of ALMPs on immigrants. The main finding of this review is that well executed integration measures can improve immigrants’ labor market attachment, speed up entering employment, and improve the quality of the attained jobs. The research suggests that the ones with the weakest attachment to the labor market can benefit the most from participating in active labor market programs, while highly educated unemployed workers may be more responsive to sanctions. Thorough analyses of ALMPs’ effects on immigrants are scarce, and the results are inconclusive. Labor market institutions, immigrant populations, and history of immigration are different even in the Nordic countries, which is why the results are not directly applicable to the Finnish context and there should be more causal research on the topic from Finland. To achieve this, the Finnish government should start experimenting with ALMPs but also implement possible new policies in a way that enables convincing causal analysis on policy effects. 35 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 R E F E R E N C E S Included Studies Andersen, L. H., Dustmann, C., & Landersø, R. (2019). Lowering Welfare Benefits: Intended and Unintended Consequences for Migrants and their Families. CReAM Discussion Paper Series. Andersson Joona, P., & Nekby, L. (2012). Intensive Coaching of New Immigrants: An Evaluation Based on Random Program Assignment. Scandinavian Journal of Economics, 114(2), 575–600. https://doi.org/10.1111/j.1467-9442.2011.01692.x Andrén, T., & Andrén, D. (2006). Assessing the employment effects of vocational training using a one-factor model. Applied Economics, 38. https://doi.org/10.1080/00036840500427577 Arendt, J. N. (2022). Labor market effects of a work-first policy for refugees. Journal of Population Economics, 35, 169–196. https://doi.org/10.1007/s00148-020-00808-z Arendt, J. N., Bolvig, I., Foged, M., Hasager, L., & Peri, G. (2020). NBER WORKING PAPER SERIES LANGUAGE TRAINING AND REFUGEES’ INTEGRATION. http://www.nber.org/papers/w26834 Åslund, O., & Johansson, P. (2011). Virtues of SIN: Can intensified public efforts help disadvantaged immigrants? Evaluation Review, 35(4), 399–427. https://doi.org/10.1177/0193841X11419282 Battisti, M., Giesing, Y., & Laurentsyeva, N. (2019). Can job search assistance improve the labour market integration of refugees? Evidence from a field experiment. Labour Economics, 61, 101745. https://doi.org/10.1016/j.labeco.2019.07.001 Bergemann, A., Caliendo, M., van den Berg, G. J., & Zimmermann, K. F. (2011). The threat effect of participation in active labor market programs on job search behavior of migrants in Germany. International Journal of Manpower, 32(7), 777–795. Bernhard, S., Gartner, H., & Stephan, G. (2008). Wage Subsidies for Needy Job-Seekers and Their Effect on Individual Labour Market Outcomes after the German Reforms. IZA Discussion Paper, 3772. Bernhard, S., & Kruppe, T. (2012). Effectiveness of further vocational training in Germany Effectiveness of further vocational training in Germany Empirical findings for persons receiving means-tested unemployment benefit. IAB Discussion Paper. Bernhard, S., & Wolff, J. (2008). Contracting out placement services in Germany Contracting out placement services in Germany Is assignment to private providers effective for needy job-seekers? IAB Discussion Paper, 5. Caliendo, M., & Künn, S. (2010). Start-Up Subsidies for the Unemployed: Long-Term Evidence and Effect Heterogeneity. IZA Discussion Paper. Clausen, J., Heinesen, E., Hummelgaard, H., Husted, L., & Rosholm, M. (2009). The effect of integration policies on the time until regular employment of newly arrived immigrants: Evidence from Denmark. Labour Economics, 16(4), 409–417. https://doi.org/10.1016/j.labeco.2008.12.006 Dahlberg, M., Egebark, J., Vikman, U., & Özcan, G. (2020). Labor market integration of low-educated refugees-RCT evidence from an ambitious integration program in Sweden. IFAU Working paper 2020:21. www.ifau.se Delander, L., Hammarstedt, M., Månsson, J., & Nyberg, E. (2005). INTEGRATION OF IMMIGRANTS The Role of Language Proficiency and Experience. Evaluation Review, 29(1), 24–41. https://doi.org/10.1177/0193841X04270230 Drange, N., & Telle, K. (2015). Promoting integration of immigrants: Effects of free child care on child enrollment and parental employment ☆. https://doi.org/10.1016/j.labeco.2015.03.006 East, C. N. (2018). Immigrants’ labor supply response to Food Stamp access ☆. Labour Economics, 51, 202–226. https://doi.org/10.1016/j.labeco.2018.01.003 Hardoy, I., & Schøne, P. (2010). Incentives to work? The impact of a “Cash-for-Care” benefit for immigrant and native mothers labour market participation. https://doi.org/10.1016/j.labeco.2010.02.008 Heinesen, E., Husted, L., & Rosholm, M. (2013). The effects of active labour market policies for immigrants receiving social assistance in Denmark. IZA Journal of Migration, 2(15). https://doi.org/10.1186/2193-9039-2-15 Heller, B., Slungaard Mumma, K., Avery, C., Deming, D., Dobbie, W., Goldin, C., Goodman, J., Kane, T., Setren, E., & Staiger, D. (2020). Immigrant Integration in the United States: The Role of Adult English Language Training. https://doi.org/10.26300/7rxa-v748 https://doi.org/10.1111/j.1467-9442.2011.01692.x https://doi.org/10.1080/00036840500427577 https://doi.org/10.1007/s00148-020-00808-z http://www.nber.org/papers/w26834 https://doi.org/10.1177/0193841X11419282 https://doi.org/10.1016/j.labeco.2019.07.001 https://doi.org/10.1016/j.labeco.2008.12.006 http://www.ifau.se https://doi.org/10.1177/0193841X04270230 https://doi.org/10.1016/j.labeco.2015.03.006 https://doi.org/10.1016/j.labeco.2018.01.003 https://doi.org/10.1016/j.labeco.2010.02.008 https://doi.org/10.1186/2193-9039-2-15 https://doi.org/10.26300/7rxa-v748 36 Publications of the Ministry of Economic Affairs and Employment 2022:27 Hohmeyer, K., & Wolff, J. (2007). A fistful of Euros Does One-Euro-Job participation lead means-tested benefit recipients into regular jobs and out of unemployment benefit II receipt? IAB-Discussion Paper, 32. Huber, M., Lechner, M., Wunsch, C., & Walter, T. (2009). Do German Welfare-to-Work Programmes Reduce Welfare and Increase Work? IZA Discussion Papers, 4090. www.econstor.eu Hwang, M.-C. (2016). The Effect of Multicultural Family Support Service: Examining Integration with Immigrant Wives in South Korea. Journal of Social Service Research, 42(5), 630–650. https://doi.org/10.1080/01488376.2016.1216917 Jahn, E., & Rosholm, M. (2013). Is temporary agency employment a stepping stone for immigrants?. Economics Letters, 118(1), 225-228. Joachim Wolff, & Anton Nivorozhkin. (2008). Start me up The effectiveness of a self-employment programme for needy unemployed people in Germany. IAB Discussion Paper, 20. Kaestner, R., & Kaushal, N. (2005). Immigrant and native responses to welfare reform. Journal of Population Economics, 18(1), 69–92. https://doi.org/10.1007/s00148-004-0185-2 Kivi, L. H., Sõmer, M., & Kallaste, E. (2020). Language training for unemployed non-natives: Who benefits the most? Baltic Journal of Economics, 20(1), 34–58. https://doi.org/10.1080/1406099X.2020.1740403 Knoef, M., & Ours, J. C. van. (2016). How to stimulate single mothers on welfare to find a job: evidence from a policy experiment. Journal of Population Economics, 29(4), 1025–1061. https://doi.org/10.1007/s00148-016-0593-0 Lang, J. (2021). Employment effects of language training for unemployed immigrants. Journal of Population Economics. https://doi.org/10.1007/s00148-021-00832-7 Lochmann, A., Rapoport, H., & Speciale, B. (2019). The effect of language training on immigrants’ economic integration: Empirical evidence from France R. European Economic Review, 113, 265–296. https://doi.org/10.1016/j.euroecorev.2019.01.008 Lopalo, M. (2019). The effects of cash assistance on refugee outcomes. Journal of Public Economics, 170, 27–52. https://doi.org/10.1016/j.jpubeco.2018.11.004 Månsson, J., & Delander, L. (2017). Mentoring as a way of integrating refugees into the labour market- Evidence from a Swedish pilot scheme. Economic Analysis and Policy, 56, 51–59. https://doi.org/10.1016/j.eap.2017.08.002 Pastore, F., & Pompili, M. (2020). Assessing the Impact of Off-the-Job and On-the-Job Training on Employment Outcomes: A Counterfactual Evaluation of the PIPOL Program. Evaluation Review, 44(3), 145–184. https://doi.org/10.1177/0193841X20966112 Richardson, K., & van den Berg, G. J. (2013). Duration dependence versus unobserved heterogeneity in treatment effects: Swedish labor market training and the transition rate to employment. Journal of Applied Econometrics, 28(2), 325–351. https://doi.org/10.1002/JAE.2263 Rønsen, M., & Skarðhamar, T. (2009). Do welfare-to-work initiatives work? Evidence from an activation programme targeted at social assistance recipients in Norway. Journal of European Social Policy, 19(1), 61–77. https://doi.org/10.1177/0958928708098524 Rosholm, M., & Vejlin, R. (2010). Reducing income transfers to refugee immigrants: Does start-help help you start? Labour Economics, 258–275. https://doi.org/10.1016/j.labeco.2009.09.004 Sarvimäki, M., & Hämäläinen, K. (2016). Integrating immigrants: The impact of restructuring active labor market programs. Journal of Labor Economics, 34(2), 479–508. https://doi.org/10.1086/683667 Svarer, M. (2011). The effect of sanctions on exit from unemployment: Evidence from Denmark. Economica, 78(312), 751–778. https://doi.org/10.1111/j.1468-0335.2010.00851.x Thomsen, S. L., & Walter, T. (2010). Temporary Extra Jobs for Immigrants: Merging Lane to Employment or Dead- End Road in Welfare? Labour, 24(SUPPL. 1), 114–140. https://doi.org/10.1111/J.1467-9914.2010.00505.X Thomsen, S. L., Walter, T., & Aldashev, A. (2013). Short-term training programs for immigrants in the German welfare system: do effects differ from natives and why? IZA Journal of Migration, 2(24). http://www.izajom.com/content/2/1/24 Ugreninov, E., & Turner, L. M. (2021). Next to Nothing: The Impact of the Norwegian Introduction Programme on Female Immigrants’ Labour Market Inclusion. Journal of Social Policy, 1–22. https://doi.org/10.1017/S004727942100043X Vikman, U. (2013). Paid parental leave to immigrants: An obstacle to labor market entrance? IFAU Working paper 2013:4. www.ifau.se Wikström, M., Kotyrlo, E., & Hanes, N. (2015). CHILDCARE REFORM: EFFECTS ON EARNINGS AND EMPLOYMENT AMONG NATIVE SWEDISH AND IMMIGRANT MOTHERS. Gender in the Labor Market (Research in Labor Economics, Vol. 42), 42, 93–129. https://doi.org/10.1108/S0147-912120150000042003 Wolff, J., & Jozwiak, E. (2007). Does short-term training activate means-tested unemployment benefit recipients in Germany? IAB Discussion Paper, 29. http://hdl.handle.net/10419/32680www.econstor.eu http://www.econstor.eu https://doi.org/10.1080/01488376.2016.1216917 https://doi.org/10.1007/s00148-004-0185-2 https://doi.org/10.1080/1406099X.2020.1740403 https://doi.org/10.1007/s00148-016-0593-0 https://doi.org/10.1016/j.euroecorev.2019.01.008 https://doi.org/10.1016/j.jpubeco.2018.11.004 https://doi.org/10.1016/j.eap.2017.08.002 https://doi.org/10.1177/0193841X20966112 https://doi.org/10.1002/JAE.2263 https://doi.org/10.1177/0958928708098524 https://doi.org/10.1016/j.labeco.2009.09.004 https://doi.org/10.1086/683667 https://doi.org/10.1111/j.1468-0335.2010.00851.x https://doi.org/10.1111/J.1467-9914.2010.00505.X http://www.izajom.com/content/2/1/24 https://doi.org/10.1017/S004727942100043X http://www.ifau.se https://doi.org/10.1108/S0147-912120150000042003 http://hdl.handle.net/10419/32680www.econstor.eu 37 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 Other References Angrist, J. D., & Pischke, J. S. (2008). Mostly harmless econometrics. Princeton university press. Åslund, O., & Engdahl, M. (2017). The value of earning for learning: Performance bonuses in immigrant language training ☆. https://doi.org/10.1016/j.econedurev.2017.11.010 Blandhol, C., Bonney, J., Mogstad, M., & Torgovitsky, A. (2022). When is TSLS Actually LATE?. University of Chicago, Becker Friedman Institute for Economics Working Paper, (2022-16). Butschek, S., & Walter, T. (2014). What active labour market programmes work for immigrants in Europe? A meta-analysis of the evaluation literature. IZA Journal of Migration, 3(1), 1-18. Card, D., Kluve, J., & Weber, A. (2018). What works? A meta analysis of recent active labor market program evaluations. Journal of the European Economic Association, 16(3), 894-931. Crépon, B., Duflo, E., Gurgand, M., Rathelot, R., & Zamora, P. (2013). Do labor market policies have displacement effects? Evidence from a clustered randomized experiment. The quarterly journal of economics, 128(2), 531-580. Filges, T., & Hansen, A. T. (2017). The threat effect of active labor market programs: A systematic review. Journal of Economic Surveys, 31(1), 58-78. Huntington-Klein, N. (2021). The effect: An introduction to research design and causality. Chapman and Hall/CRC. Westgate MJ (2019). revtools: Tools to Support Evidence Synthesis. R package version 0.4.1, https://CRAN.R-project.org/package=revtools https://doi.org/10.1016/j.econedurev.2017.11.010 https://CRAN.R-project.org/package=revtools 38 Publications of the Ministry of Economic Affairs and Employment 2022:27 Appendix A Date Source Search Terms Filters 27.10.2021 Econlit, Web of Science (immigr* OR migrant OR refugee OR non-native OR asylum) AND ("active labor market" OR integrat* OR "wage subsidy" OR “job search assistance” OR monitor* OR “language skills” OR coaching OR reform OR intervention OR training OR sanction) AND (experiment OR treatment OR causal* OR quasi* OR intervention OR rct OR “randomized controlled trial” OR “treatment group” OR “control group” OR "propensity score matching" OR “instrumental variable” OR “regression discontinuity design” OR "difference-in-differences" OR "differences-in-differences" OR “difference in differences” OR “event study” OR matching OR discontinuity OR impact OR “duration model” OR effect) AND (employ* OR “labor market” OR “labour market” OR unemploy*) 27.10.2021 Scopus (immigr* OR migrant OR refugee OR non-native OR asylum) AND ("active labor market policy" OR integration OR "wage subsidy" OR “job search assistance” OR monitor* OR “vocational training” OR “language skills” OR coaching OR reform OR intervention) AND (experiment OR treatment OR causal* OR quasi* OR intervention OR rct OR “randomized controlled trial” OR “treatment group” OR “control group” OR "propensity score matching" OR “instrumental variable” OR “regression discontinuity design” OR "difference-in-differences" OR "differences-in-differences" OR “difference in differences” OR “event study” OR matching) AND (employment OR “labor market” OR “labour market” OR unemployment) 27.10.2021 Jstor: (((immigrant OR migrant OR refugee) AND (active labor market OR almp OR integration)) AND ("treatment group" OR "control group")) 1.11.2021 Web of Science (((ALL=(immigr* OR migrant OR refugee OR non-native OR asylum)) AND ALL=("active labor market" OR integrat* OR "wage subsidy" OR “job search assistance” OR monitor* OR “language skills” OR coaching OR reform OR intervention OR training OR sanction)) AND ALL=(experiment OR treatment OR causal* OR quasi* OR intervention OR rct OR “randomized controlled trial” OR “treatment group” OR “control group” OR "propensity score matching" OR “instrumental variable” OR “regression discontinuity design” OR "difference-in-differences" OR "differences-in-differences" OR “difference in differences” OR “event study” OR matching OR discontinuity OR impact OR “duration model” OR effect)) AND ALL=(employ* OR “labor market” OR “labour market” OR unemploy*) WoS Category: Economics 39 Publications of the Ministry of Economic Affairs and Employment 2022:27 Publications of the Ministry of Economic Affairs and Employment 2022:27 Date Source Search Terms Filters 5.11.2021 CES Working Papers Topics: Employment: All, Years 2005-2021 5.11.2021 IFAU “active labor market polic” and “immigra” 2005- 5.11.2021 IFAU “immigra” 2005- 5.11.2021 IZA “active labor market policy” 2005- 8.11.2021 NBER Working papers “("active labor market policy" OR training OR integrat*) AND immigra*” Year: 2005-, Topics: “Labor Economics” and “Unemployment and Immigration” 40 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Appendix B Table B1. Secondary studies using non-experimental or non-quasi-experimental approaches (matching etc.) Study, * if published Country Policy/Treatment Policy target group Method Difference between groups (employment) 1 Kivi et al. (2020) * Estonia Language training for unemployed Immigrants Matching Significant pos. 8-% points difference two years after 2 Pastore & Pompili (2020) * Italy On-the-job or off-the-job training General Matching Significant pos. 10-20-% points difference two years after 3 Månsson & Delander (2017) * Sweden Mentoring Immigrants Matching Employment difference not significant 4 Hwang (2016) * South Korea Multicultural Family Support Service for Immigrant Wives Immigrants Matching Employment difference not significant 5 Richardson & Van den Berg (2013) * Sweden Vocational employment training program General Duration model+ Employment difference not significant (main model) 6 Heinesen et al. (2013) * Denmark ALMPs (subsidized emp., direct employment prog., other prog.) General Duration model+ Significant positive different hazard rate to employment 7 Thomsen et al. (2013) * Germany Short-term training (Four different types) General Matching Significant pos. 10 % diff. one year after (aptitude tests) 8 Bernhard & Kruppe (2012) Germany Subsidized vocational training General Matching Significant pos. 8-% points difference after two years 9 Bergemann et al. (2011) * Germany Ex-ante effects of participation in ALMP General Matching No difference compared to similar native groups 10 Caliendo & Künn (2010) Germany Start-up subsidies for unemployed General Matching Mixed results, indications that more effective for natives 41 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Publications of the M inistry of Economic A ffairs and Employment 2022:27 Study, * if published Country Policy/Treatment Policy target group Method Difference between groups (employment) 11 Thomsen & Walter (2010) * Germany Subsidized employment (Temporary Extra Jobs) General Matching No significant positive difference 12 Clausen et al. (2009) * Denmark ALMPs and language training General Duration model+ Private subsidized employment relates to 14-24 weeks shorter duration 13 Huber et al. (2009) Germany Welfare-to-work program General Matching No significant difference 14 Rønsen et al. (2009) * Norway ALMP (different welfare-to-work policy) General Matching No significant difference 15 Bernhard & Wolff (2008) Germany Temporary assignment to private placement services General Matching Significant pos. 4-5-% points diff. for men in West Ger. 16 Bernhard et al. (2008) Germany Wage subsidy General Matching Significant pos. 32-35-% points difference 17 Wolff & Nivorozhkin (2008) Germany Start-up subsidies for unemployed General Matching Significant pos. 32-35-% points difference 18 Hohmeyer & Wolff (2007) Germany Workfare program ("one-euro-job") General Matching Significant pos. 22-32-% points difference 19 Wolff & Jozwiak (2007) Germany Short-term training (classroom and within company) General Matching Significant pos. 12-18-% points difference 20 Andrén & Andrén (2006) * Sweden Vocational training General One-factor model Mixed results 21 Delander et al. (2005) * Sweden Language training and workplace training (Sesame project) Immigrants Matching No significant difference in hazard rates (average) 42 Publications of the Ministry of Economic Affairs and Employment 2022:27 Appendix C. Suomenkielinen abstrakti Maahanmuuttaneiden työllistyminen ja onnistunut integroituminen osaksi yhteiskuntaa on keskeinen haaste julkisen sektorin näkökulmasta. Väestön ikääntyminen ja yksityisen sektorin osaajapula tulevat rajoittamaan Suomen