In recent years there have been many posts, articles and reports on how AI and automation will shape the future of work. Depending on the author’s perspective or agenda, these pieces go one of two ways: either the new technology will destroy jobs and have devastating effects on the labor market, or it will create a better, brighter future for everyone by destroying only the boring jobs and generating better, much more interesting ones. As always, the truth probably lies somewhere between these two extremes. In this post, we want to take a more nuanced view by discussing the most common arguments and claims and comparing them with the facts. But before we get into this, let us first clarify what the AI-driven digital transformation is. In a nutshell, it is all about automation, using AI technology to complete tasks that we do not want humans to perform, or that humans cannot perform. Just as we did in the past, in the first, second and third industrial revolutions.

From stocking looms to AI art

With each of these revolutions came the fear that human workers would be rendered obsolete. So why do we want to automate? Even though in some cases, inventors were, and still are, simply interested in the feat of the invention itself, more often than not an invention or development was driven by business interests. As is widespread adoption. And no matter which era, businesses rarely have other goals than staying competitive and raising profits. 16th century stocking looms were invented to increase productivity and lower costs by substituting human labor. Steam-powered machines in 19th century mills and factories and farm machinery were used for the same reason. Robots in vehicle manufacture in the second half of the 20th century – ditto. Whether the technology is tractors, assembly lines, or spreadsheets, the first-order goal was to substitute human musculature by mechanical power, human handiwork by machine-consistency, and slow and error-prone “humanware” by digital calculation. But so far, even though many jobs were lost to automation, others have been created. Massively increased production called for jobs related to increased distribution. With passenger cars displacing horse-powered travel and equestrian occupations, and increasing private mobility, jobs were instead created in the expanding industry of roadside food and accommodation. Increasing computational power used to replace human tasks in offices also led to entirely new products and the gaming industry. And the rising wealth and population growth accompanying such developments led to increased recreational and consumption demands, boosting these sectors and creating jobs – albeit not as many as one may think, as we will see below. However, we cannot simply assume that the current revolution will follow the same pattern and create more jobs and wealth than it will destroy just because this is what happened in the past. Unlike mechanical technology and basic computing, AI technologies not only have the potential to replace cheap laborers, say, with cleaning or agricultural robots. They have also begun outperforming expensive workers such as pathologists diagnosing cancer and other medical professionals diagnosing and treating patients, and are also touching on creative tasks such as choosing scenes for movie trailers or producing digital art. Of course, we should also not simply assume a dystopian future of less jobs and sinking wealth. But we must keep in mind that currently, in many cases it is more cost effective to replace expensive workers with AI solutions than cheap laborers such as textile workers in Bangladesh.

So, working towards a more differentiated view, let us take a look at the currently most common claims and how they stand up to closer scrutiny.

Claim 1. AI will create more/less jobs than it destroys

This is the main argument put forth in utopian/dystopian scenarios, including reports by WEF (97m new jobs vs. 85m displaced jobs across 26 countries by 2025), PwC (“any job losses from automation are likely to be broadly offset in the long run by new jobs created”), Forrester (job losses of 29% by 2030 with only 13% job creation to compensate) and many others. Either way, any net change can pose significant challenges. As BCG states in a recent report on the topic, “the net number of jobs lost or gained is an artificially simple metric” to estimate the impact of digitalization. A net change of zero or even an increase of jobs could cause major asymmetries in the labor market with dramatic talent shortages in some industries or occupations and massive worker surplus and unemployment in others. On the other hand, instead of causing unemployment – or at least underemployment, less jobs could also lead to more job sharing and thus shorter work weeks. Then again, although this may sound good in theory, it also raises additional questions: How will pay and benefits be affected? And who reaps the bulk of monetary rewards? Companies? Workers? The government? It is admittedly too soon to see the effects of AI adoption so far regarding overall employment or wages. But past outcomes, i.e., of previous industrial revolutions, do not guarantee similar outcomes in the future. And even those outcomes show that job and wealth growth were not necessarily as glorious as they are often portrayed. The ratio of employment to working age population has remained fairly constant in OECD countries since 1970, rising only from just over 64% to just under 69%.[1] Much of this increase can be attributed to higher labor participation rates, especially of women. And the increased wealth is clearly not evenly distributed, e.g., in the US:

 

Source: Economic Policy Institute, https://www.epi.org/publication/charting-wage-stagnation/

 

Source: Economic Policy Institute, https://www.epi.org/publication/charting-wage-stagnation/

 

There are simply no grounds to assume that AI and automation will automatically make us wealthier as a society or that the increased wealth will be distributed evenly. We should thus be equally prepared for more negative scenarios and discuss how to mitigate the consequences. For instance, would it be acceptable to treat AI processes like human labor? If so, we could consider taxing them to support redistribution of wealth or to finance training or benefits and pensions for displaced workers.

In addition, these estimates should be questioned on a basic level. Who can confidently say that this job will decline? How can we know what kind of jobs there will be in future? None of these projections are truly reliable or objective – they are primarily based on some group of people’s opinions. For instance, the WEF’s Future of Jobs Report, one of the most influential reports on this topic, is based on employer surveys. But it is simply naive to think anyone, let alone a cadre of arbitrary business leaders, can have a confident understanding of which jobs and skills will be required in the future. One should not expect more from this than from fortune telling at a fair. Just take a look at the predictions about cars in the early 19th century, remote shopping in the 1960s, cell phones in the 1980s, or computers since the 1940s. So many tech predictions have been so utterly wrong, why should this change now? And yet, such predictions are a key element in the estimates for the “future of work”.

Fact is, scientifically sound research on this topic is extremely scarce. One of the few papers in this area studied the impact of AI on labor markets in the US from 2007 to 2018. The authors (from MIT, Princeton and Boston University) found that greater AI exposure within businesses is associated with lower hiring rates, i.e., that AI adoption has so far been concentrated on substitution as opposed to augmentation of jobs. The same paper also finds no evidence that the large productivity effects of AI will increase hiring. Some people may be tempted to say that this supports the dystopian view. However, we must also note that this study is based on online vacancy data, and thus the results should be treated with caution, as we explained in detail in one of our other posts. In addition, due to the dynamics of technological innovation and adoption, it is almost impossible to extrapolate and project such findings to make robust predictions for future developments.

And on a more philosophical note, what would it mean for human existence if we worked substantially less? Work is ingrained in our very nature; it is a defining trait.

Claim 2: Computers are good at what we find hard and bad at what we find easy

Hard and easy for who? Luckily, we do not all have the same strengths and weaknesses, so clearly, we do not all find the same tasks “easy” and “hard”. This is just yet another extremely generalizing statement based on completely subjective judgement. And if it were true, then most people would probably consider repetitive tasks as typically easy, or at least easier. This directly contradicts the next claim:

Claim 3: AI will (only) destroy repetitive jobs and will generate more interesting, higher-value ones.

The WEF states that AI will automate repetitive tasks like data entry and assembly line manufacturing, “allowing workers to focus on higher-value and higher-touch tasks” with “benefits for both businesses and individuals who will have more time to be creative, strategic, and entrepreneurial.” BCG talks of the “shift from jobs with repetitive tasks in production lines to those in the programming and maintenance of production technology” and how “the removal of mundane, repetitive tasks in legal, accounting, administrative, and similar professions opens the possibility for employees to take on more strategic roles”. The question is, who exactly benefits from this? Not every worker who is able to perform repetitive tasks has the potential to take on strategic, creative and entrepreneurial roles, or program and maintain production technology. It is simply a fact that not everyone can be trained for every role. More satisfying, interesting tasks for intellectuals (such as the advocates of a brighter future of work thanks to AI) may simply be too challenging for the average blue-collar worker whose job – which may well have been perfectly satisfying to them – has just been automated. And not every white-collar worker can or wants to be an entrepreneur or strategist. Also, what exactly does “higher-value” mean? Who benefits from this? The new jobs created so far, like Amazon warehouse workers, or Uber and Postmates drivers, are not exactly paying decent, secured living wages. And since the early 1970s, businesses have demonstrated a clear disinterest in sharing the added value from productivity gains with workers:

 

Source: Economic Policy Institute, https://www.epi.org/publication/charting-wage-stagnation/

 

On the other hand, a vast number of the AI applications that are already available perform higher- to highly-skilled tasks based on data mining, pattern recognition and data analysis: diagnosis and treatment of medical conditions, customer service chatbots, crop optimization and farming strategies, financial or insurance advising, fraud detection, scheduling and routing in logistics and public transport, market research and behavioral analysis, workforce planning, product design, and much more. The full effect of these applications on the job market is not yet clear, but they are certainly not only removing mundane, repetitive tasks from job profiles.

Claim 4: We (just) need to up-/reskill workers.

While we certainly do not disagree with this statement in general, it is often brought up as a more or less simple remedy to prepare for the future AI-driven shifts in the labor market and “embrace the positive societal benefits of AI” (WEF). Fact is, this comes with several caveats which make it a far from simple solution.

First off, we cannot repeat enough that it is not possible to predict the “future of work” reliably, especially which jobs really will be in demand in future and which will not. Also, based on the effects of the previous industrial revolutions and current research, it is highly likely that widespread adoption of AI will introduce new jobs with profiles that we cannot anticipate at the moment. This means we need to equip current and future professionals with the skills necessary for jobs that we currently know nothing about. One often suggested way to work around this issue is to encourage lifelong learning and promote more adaptable and short-term forms of training and education. This is certainly a valid option and clearly on the rise. However, there are several aspects to keep in mind. For instance, 15–20% of the US and EU adult population[2] have low literacy (PIAAC level 1 or below). This means that they have trouble with tasks such as filling out forms or understanding texts about unfamiliar topics. How can these people be trained to succeed at “more complex and rewarding projects” if they cannot read a textbook, navigate a manual or write a simple report? In addition, around 10% of full-time workers in the US and EU are working poor.[3] These people typically have neither the time, resources, nor support from employers for lifelong learning and thus no well-informed access to efficient, targeted and affordable (re)training.

By the time such issues have been addressed, many of these workers may have already missed the boat. In 2018, US employers estimated that more than a quarter of their workforce would need at least three months of training just to keep pace with the necessary skill requirements of their current roles by 2022.[4] Two years later, that share has more than doubled to over 60%, and the numbers are similar across the globe.[5] In addition, even before the post-Great Recession period, only roughly 6 in 10 displaced US workers were re-employed within 12 months in the 2000 to 2006 period.[6] In 2019, this rate was the same in the EU.[7] With increasingly rapid changes in skills demands, combined with lack of time and/or resources for vulnerable groups such as the working poor and workers with low literacy, not to mention lacking safety nets and targeted measures in underfunded workforce development systems, the prospects for these workers are highly unlikely to improve.

Moreover, the pandemic has massively accelerated the adoption of automation and AI in the workplace in many sectors. Robots, machines and AI systems have been deployed to clean floors, take temperatures or food orders, replace employees in dining halls, toll booths or call centers, patrol empty real estate, increase industrial production of hospital supplies and much more within an extremely short period of time. In the past, new technology was deployed gradually, giving employees time to transition into new roles. This time, employers scrambled to replace workers with machines or software due to sudden lockdown or social distancing orders. This is a crucial difference to the preceding industrial revolutions. Many workers have been cut loose with simply not enough time to retrain. Similarly disruptive events may well occur in the future – be it another pandemic or a technological breakthrough – and as a society, we need to be prepared for these events and provide affected workers with swift, efficient and above all realistic support.

Claim 5: Employers should view up- and reskilling as an investment, not as an expense.

If a company replaces all of its cashiers by robots, why would they want to reskill the newly redundant workers? Even governments have a hard time taking this stance on training and education. Many countries focus primarily on college or other education for young workers rather than retraining job seekers or employees. For instance, the US government spends 0.1% of GDP to help workers navigate job transitions, less than half what it spent 30 years ago – despite the fact that skills demand is changing much faster than it did three decades ago. And the vast majority of businesses is primarily interested in maximizing profits – that is just how our economy works. Remember: we live in a world where even sandwich makers and dogwalkers are forced to sign noncompete agreements to prevent them from getting a raise by threatening to move to a competitor for higher pay.

Well-performing conversational software could enable a company to take a 1,000-person call center and run it with 100 people plus chatbots. A bot can respond to 10,000 queries in an hour, far higher than any realistic volume even the most efficient call-center rep could handle. In addition, a chatbot does not fall ill, need time off work, or ask for perks and benefits. They make consistent, evidence-based decisions and do not steal from or defraud their employers. So, if the quality of this software is sufficient and the price is right, there would most probably be an uproar amongst shareholders if a company did not go for this offer. After all, a solution that increases efficiency and productivity while lowering expenses is the ideal of businesses of our time. So, if this company doesn’t opt for it, its competition will. And despite the “tech for social good” propaganda we constantly hear from Silicon Valley, most companies are simply not interested in the future of soon-to-be-ex workers.

Beyond the bubble

The bottom line is that we cannot afford to overdramatize or simply reassure ourselves that there will be enough jobs to go round, or we will constantly be playing catch up. Most of the commonly cited problems or solutions tend to be discussed within the academic or high-income bubble of researchers, tech entrepreneurs and policy makers, mixed with a substantial amount of idealism. But to get ahead of these developments that – for good or for bad – have vast potential to completely transform our labor markets and society, we need to look beyond our bubble and design realistic strategies for the future based on facts and objective data.

 

[1]    https://stats.oecd.org/Index.aspx?DatasetCode=LFS_SEXAGE_I_R#
[2]   US: https://www.libraryjournal.com/?detailStory=How-Serious-Is-Americas-Literacy-Problem
        EU: http://www.eli-net.eu/fileadmin/ELINET/Redaktion/Factsheet-Literacy_in_Europe-A4.pdf
[3]   US: https://www.policylink.org/data-in-action/overview-america-working-poor
        EU: http://www.europeanrights.eu/public/commenti/BRONZINI13-ef1725en.pdf
[4]   The Future of Jobs Report 2018, World Economic Forum, 2018.
[5]   The Future of Jobs Report 2020, World Economic Forum, 2020.
[6]   Back to Work: United States: Improving the Re-employment Prospects of Displaced Workers, OECD, 2016.
[7]   https://skillspanorama.cedefop.europa.eu/en/dashboard/long-term-unemployment-rate?year=2019&country=EU#1

 

To follow the trend of future work, upskilling and reskilling, and digital transformation, we are posting a series of articles to analyze government policies and practices to learn how countries are taking strategies to build their workforce for these challenges. In the previous article, we examined how Singapore is helping mid-career PMETs to switch to the tech sector. Our second stop is Saudi Arabia.

The Kingdom of Saudi Arabia has shown strong commitment to the implementation and development of AI as it seeks to diversify the economy, reduce the dependence on oil and shift away from public sector-driven welfare within a strategic framework called Saudi Vision 2030. According to estimates by PwC, the contribution of AI to the national economy in Saudi Arabia will reach US$135.2 bn by 2030, around 12.4% of its total GDP.

Saudi Vision 2030 comprises a host of ambitious targets for the labor market, including lowering the unemployment rate from 11.6% to 7% [1], increasing female participation from 22% to 30% [2], and bolstering the private sector with a special emphasis on small and medium-sized enterprises (SMEs) to increase their contribution to GDP from 20% to 35% by 2030 [3].

Under Saudi Vision 2030, Saudi Arabia is spending heavily on the communication and information technology (ICT) sector. The International Data Corporation (IDC) predicts that spending on IT in Saudi Arabia will exceed $11 billion in 2021. In addition to Saudi Vision 2030, the Kingdom has launched its National Strategy for Data and Artificial Intelligence (NSDAI) and last year signed a series of partnership agreements with international tech companies last year to advance AI in Saudi Arabia. One of the key pillars of the NSDAI is to build an AI-ready workforce in the Kingdom.

The growing industry is putting high pressure on a domestic labor market that lacks a sufficiently large and experienced national ICT workforce. Saudi Arabia is highly reliant on expat workers, and many of its professionals, including most in ICT, are sourced internationally. This has hampered many private companies and especially SMEs due to high recruiting costs. Furthermore, as skilled talents are increasingly in short supply globally, Saudi Arabia needs to focus on its young domestic population.

With 60% of the population under 30 years of age in 2020, Saudi Arabia has one of the world’s youngest populations. However, youth employment remains low in the Kingdom, and 16% of Saudi youth between 20 and 24 are classified as NEET (not in education, employment or training) [4]. Several reports indicate that there is a notable gap between the future-proof skills requested in the labor market and those possessed by the young people, and that the Kingdom is lacking an effective skill matching mechanism. Current skills gaps include STEM skills (science, technology, engineering and math), soft skills, and industry-specific international skills due to a lack of vocational training [5]. Therefore, part of the NSDAI agenda is to provide professional AI training to Saudi university students, researchers and developers, as well as up- and reskilling opportunities that enable Saudis to utilize AI and data in both the public and private sectors. Programs including the National AI Capability Development Program and the National AI Talent Cultivation and Onboarding Program have already been set up by the government as a collaboration between tech partners and education providers to attract, develop and retain AI talent in Saudi Arabia with a target of creating 20,000 AI and data specialists and experts by 2030.

As countries embark on the journey of digital transformation, their governments and public employment services (PES) seek suitable solutions to support them in the increasingly important role they play in job matching, enhancing employability, addressing skill gaps and aligning education offerings with market needs. To learn about how JANZZ.technology assists PES in tackling these challenges, please visit our product site for PES or contact us at info@janzz.technology

 

[1] Readiness for the Future of Work, ATKearney and MISK.

[2] KSA Vision 2030: Strategic Objectives and Vision Realization Programs 61.

[3] KSA National Transformation Program Delivery Plan – 2018-2020

[4] International Labour Organization, ILOSTAT database.

[5] Building the talent pipeline in Saudi Arabia, City&Guilds Group

In recent years, online job and candidate searches have become increasingly important and a growing number of CVs and resumes is now available in digital form. Despite recurring announcements of their demise, and even if the delivery format and selection of information has changed, the need for details on a candidate’s professional background persists. Thus, more and more businesses are turning to automation tools to handle the increasing volume of CVs and resumes. However, because these documents are not standardized, the available recruitment automation and resume parsing tools are faced with significant challenges and are known to screen out good candidates. So, an essential and far from trivial question is how to process this information to produce smart, meaningful data that can be utilized by recruiting tools to pinpoint the best candidates for the vacancy at hand.

Even with the rapid advance of AI-based systems, most ATS resume parsing algorithms are outdated and unintelligent, often causing essential information to get distorted or lost. Candidates are thus advised to submit a standardized resume that is neither visually appealing, nor shows any personality, to avoid being sorted out by the system. This is in stark contrast to the wealth of online advice on how to stand out from the crowd with an appealing modern resume. It also undermines efforts to improve the candidate experience. But there seems to be a general consensus that a trade-off is inevitable: either hirers choose to obtain quality resume information in the first step of the recruiting process – and run the risk of disgruntling best-in-class candidates. Or they improve the candidate experience by not asking too many questions and then simply make do with potentially jumbled parsed resume information – running the risk of great candidates falling through the ATS.

Here at JANZZ.technology, we refuse to ignore these challenges and simply hand the problem off to applicants. Our mission is to help improve both the recruiting and the candidate experience by providing efficient application processes whilst ensuring the candidates’ freedom of individual expression. With our cutting-edge CV/resume parser, we already have the text processing down, using strategies from deep learning models trained specifically for resume content to semantic technologies that translate the myriad variations in occupational jargon to a common vocabulary. As for visual aspects such as formatting, layout and graphical representations, we are currently very actively engaged in the R&D phase, developing pioneering technology to tackle this.

To find out more about the challenges and latest advances in CV and resume parsing – and to take an intriguing walk through the history of the CV – read our white paper:

A resume of CV parsing. Great candidate experience vs. ATS optimization– is a trade-off truly inevitable?

And if you want to use our state-of-the-art semantic parsing tool and benefit from the top quality in multiple languages, all services are also available via API and can be easily integrated into existing ATS or platforms. For a demo or quote, feel free to contact us at info@janzz.technology

The current worldwide pandemic has catapulted the labor market into a state of unprecedented turbulence. According to the OECD, the impact on jobs just within the first three months has been 10 times that of the 2008 financial crisis. Entire sectors such as hospitality, civil aviation and cultural sectors have been hit hard, resulting in mass job losses and collapsing self-employed incomes. On the other hand, e-commerce and supermarkets, courier and logistics services, manufacturers of food or hygiene products, pharmaceuticals and others have thrived, creating new opportunities by dramatically increasing their workforce. Even if some of these jobs are only temporary, they can be lifesavers for those in need of income.

Amidst these turbulences, Public Employment Services (PES) have faced a historical stress test, being inundated with a number of new jobseekers far beyond what their often outdated systems are typically designed for – if such systems are in place at all. With youth struggling due to the closing down of entry-level jobs and apprenticeships, and low-paid workers, women, ethnic minorities as well as self-employed and informal workers among the hardest hit by the crisis, existing vulnerabilities have been exposed and inequalities amplified. Now more than ever, PES must find new ways to best serve their people in this crisis and in future. These vulnerable groups need to be reconnected with good jobs as quickly as possible to avoid potentially long-lasting scarring effects. And PES need to be prepared for further shifts and turbulences in the labor market with innovative digital solutions that help strengthen labor market resilience by ensuring efficiency and scalability as well as providing creative placement solutions and valuable, timely insights – into the labor market and for jobseekers and employers.

Even if a country’s PES is only just being built up, now is the right time to initiate a digital transformation. In fact, there may never be a better time – especially for countries that are just getting started. A well-designed solution does not require a perfect starting point. It does not need a lot of in-house data or even a well-organized PES. It performs well in markets with only few highly skilled professionals and supports transitions from the informal to the formal economy. In addition, choosing a mobile-first approach geared towards self-service features over a cumbersome expert system plays to the strengths of one of the most affected groups of the pandemic: young people, who are used to taking matters into their own hands, finding information and discovering their options with their devices. The digital transformation has begun and those who want to be part of it must act now.

To ensure effective digitalization in these challenging times, PES should look for solutions with:

Full profile matching based on ontologies for a wider variety of fitting placement solutions

Instead of simply comparing job titles, jobseekers can be matched based on their full profile of skills, education, previous experience and many more relevant criteria. This will particularly help those who need to seek new lines of work due to collapsing economic sectors, where matching on job title alone is ineffective. In addition, using labor market-specialized ontologies enriched with country-specific content helps identify hidden skills based on education and experience. This can substantially improve jobseeker profiles and thus broaden the search for suitable jobs and candidates, whilst significantly increasing the accuracy of matches.

Searchable jobseeker profiles and simple recruitment processes for enhanced visibility and virtual mobility

Providing jobseekers with a platform to present themselves with a searchable, well-structured profile enhances their visibility and gives them the opportunity to be found by potential employers. To avoid bias, the open profile should only contain professional information. Integrating the first steps of selection and recruitment process into the system reduces the need to travel in person until a real opportunity presents itself. These features improve both visibility and virtual mobility for jobseekers, which is especially important for vulnerable groups such as low-paid or informal workers and minorities and in times of increased remote working and hiring. Easy-to-use recruitment processes also encourage smaller businesses to transition from informal candidate search, e.g., by word of mouth, to online vacancy posting, making their business and open positions visible to a wider pool of jobseekers.

Non-discriminatory, explainable matching for unbiased and transparent results

Matching processes must be explainable and auditable to ensure transparency and accountability. Moreover, solutions should be designed in a way to guarantee that by default the best candidate with the best aptitude in all individual criteria achieves the best match – regardless of gender, ethnicity, disability or other personal characteristics. This ensures that all jobseekers have equal opportunities, including youth, women and minorities.

Gap analysis for targeted job and career pathing

In times of dramatic shifts in the labor market, many jobseekers are required to identify new lines of work and change professions entirely. By determining the closest match to currently available positions and identifying missing skills, training or other relevant criteria, gap analyses can help find a path out of the employment crisis. Based on real-time labor market data and comprehensive ontologies, they can be used to counsel individual jobseekers, or even to redirect entire workforces from disappearing professions or sectors.

Smart labor market intelligence to identify real-time shifts and labor market shocks

In a turbulent labor market environment with fast and unpredictable changes, smart labor market intelligence delivered in real time as well as well-designed intelligence management tools can make the crucial difference. Processed with a powerful labor market ontology, these data can provide better, more accurate and timely insights – key to effective management and rapid response.

JANZZ systems provide all of the above features and more. We cannot create jobs, but we can help countries smooth out the effects of COVID-19 in labor markets and guide PES in a digital transformation to become more efficient and more sustainable. We can support the transition from guessing to knowing what it will take to give employment back to as many people as possible. For instance, PES can leverage our integrated labor market solution JANZZilms! to take action in key areas and rapidly respond with timely strategies that have real impact.

 

 

 

Since the beginning of March 2020, we have been able to prove how powerful and scalable our systems are. In a large European country, almost 10% of the working population has been forced to register with the unemployment office due to the impact of the pandemic. At the beginning of the crisis, the system, which was designed for about 30,000 registrations per year, was saturated with almost 400,000 registrations within just a few weeks. Although nobody had anticipated a scenario like this during the planning phase, our systems processed almost ten times the volume of transactions without any problems. Performance and stability of the systems were always fully maintained in these important times. In addition, thanks to intuitive and intelligent design, the national PES was able to both increase job counseling capacities and reduce average time to market reintegration. This way, we were able to make a valuable contribution to the government’s efforts to quickly and efficiently register, counsel and reintegrate almost 400,000 job seekers in the country.

Quick and efficient implementation – with no surprises

Our agile methods have led time and again to outstanding products – developed on time and on budget. The standard solutions can be realized in 120 – 180 days, or 90 days in a language we have already deployed before. This provides great value in terms of implementation, operation and maintenance. Moreover, prices are fixed over several years, ensuring financial predictability. For instance, the full bundle solution JANZZilms! including all components such as matching and gap analysis, profiler, ontology, several languages, parsing, dashboards and much more, with up to one million active/concurrent users costs around USD 1 per user and year after implementation. As a complete SaaS solution, no further investments for the systems and hardware, regular maintenance, SLA’s or updating processes etc. are necessary. For larger systems with up to 5 million users in all roles (e.g., PES counsellors, job seekers, companies and job providers, third party providers like education etc.) the price drops to around 60-70 cents per user and year. With even larger systems, the price quickly falls below 50 cents per year.

Moreover, our solutions run in secure, GDPR-compliant cloud environments that are perfectly suited for any IT infrastructure including simpler setups in emerging markets. They also offer great accessibility for all users – from digital natives to tech newbies, on mobile and small screen devices and with slow internet connections.

For more information on how our services and solutions can help strengthen your labor market resilience, visit our product site for PES or contact us at info@janzz.technology. If not now, then when?

This is the third in a series of posts about skills. If you haven’t already, read the other posts first:
Cutting through the BS and Sorry folks, but “Microsoft Office” is NOT a skill.

In the second post of this series, we discussed skills and the issues around defining and specifying them. Assuming we can reach some kind of common understanding of this valuable new currency, the next step is to find a way to generate meaningful skills and job data.

 

Shaky data – shaky results

Big data from online job platforms or professional networking sites can yield a wealth of information with a much higher granularity than the usual data gathered by national statistics offices in surveys – especially regarding skills. One reason is that, unlike printed advertisements, employers do not have to pay by space for online job postings and thus can provide more detailed information on the knowledge and skills they require. This online data also allows for a much larger sample to be monitored in real time, which can be highly valuable for analysts and policy makers to develop a timely and more detailed understanding of conditions and trends on the labor market.

However, when working with the data that is available online, such as online job advertisements (OJA) or professional profiles (e.g., LinkedIn profiles), we need to be clear on the fact that this data is neither complete nor representative and therefore any results must always be interpreted with caution. Not only because of the obvious fact that the results will be distorted, but more importantly because of the implications. Promoting certain skills based on distorted data can be harmful to the labor market: if workers focus on obtaining these skills – which by nature tend to be derived from data biased towards high-skilled professionals in sectors such as IT and other areas involving higher education – they are less likely to opt for career paths involving other skills that actually are in high demand, e.g., vocational careers in skilled trades, construction, healthcare, manufacturing, etc. Despite the fact that digitalization will primarily affect better educated workers with high wages in industrialized countries, simply because it is much easier to digitalize or automate at least some of the tasks in these jobs than those in many blue-collar and vocational occupations such as carpentry, care work, etc. The last thing any labor market policymaker would want is to accentuate the already critical skill gap in this area. Or create an even tighter labor market for certain professions, say, IT professionals [1]. Similarly, education providers seeking to align their curricula with market demand need reliable data so as not to amplify skill gaps instead of alleviating them. And yet, a growing number of PES are relying on this often shaky data for decision making and ALMP design.

For instance, there are several projects that aim to gather and analyze all available OJA from all possible sources in a given labor market and use these aggregated data to make recommendations including forecasts of future employability and skills demand. But the skills are typically processed and presented without any semantic context, which can be extremely misleading.

Challenges of OJA data

In 2018, the European statistical system’s ESSnet Big Data project issued a report [2] on the feasibility of using OJA data for official statistics. Their conclusion was: «the quality issues are such that it is not clear if these data could be integrated in a way that would enable them to meet the standards expected of official statistics.»

Let us take a look at some of the basic challenges of OJA data.

1. Incomplete and biased: Not all job vacancies are advertised online. A significant proportion of positions are filled without being advertised at all (some say around 20%, others claim up to 85% of vacancies). Of those that are advertised, not all are posted online. CEDEFOP reported that in 2017 the share of vacancies published online in EU countries varies substantially, ranging from almost 100% in Estonia, Finland and Sweden down to under 50% in Denmark, Greece, and Romania. [3] In addition, some types of jobs are more likely to be advertised online than others. And large companies or those with a duty to publish vacancies are typically statistically overrepresented while small businesses, who often prefer other channels such as print media, word of mouth, or signs in shop windows, are underrepresented. Another relevant point is that certain markets are so dried out that advertising vacancies is just not worthwhile, and specialized headhunters are used instead. In summary, this means that OJA data not only fail to capture many job vacancies, but are also not representative of the overall job market. [4]
2. Duplicates: In most countries, there is no single source of OJA data. Each country has numerous online job portals, some of which publish only original ads, others that republish ads from other sources, hybrid versions, specialized sites for certain sectors or career levels, etc. So, to ensure adequate coverage, OJA data generally need to be obtained from multiple sources. This inevitably leads to many duplicates, which must be dealt with effectively in order to reliably measure labor market trends in the real world. For instance, in a 2016 project the UK national statistics institute (NSI) reported duplicate percentages of 8-22% depending on the portal, and an overall duplication rate of 10%. [5] In the ESSnet Big Data project, the Swedish NSI identified 4-38% duplicates per portal and 10% in the merged data set [6].
3. Inconsistent level of detail: Certain job postings provide much more explicit information on required skills than others, for instance depending on the sector (e.g., technical/IT) or country (e.g., due to legislation or cultural habits). Moreover, implicit information is recorded only to a limited extent and is statistically underrepresented, despite its high relevance. One reason for this is that US data providers often fail to recognize how uniquely detailed OJA are in the US, thus assuming that this is true everywhere and basing their methods on this assumption. However, this is far from correct. For instance, a job description like the one below, which is fairly typical in the US, will often be condensed to «carry out all painting work in the areas of maintenance, conversions and renovations; compliance with safety and quality regulations; minimum three years of experience or apprenticeship» in European countries. Moreover, in job ads like this, many of the required skills must be derived from the listed tasks or responsibilities. This shows just how important it is to extract implicit information.

 

 

So, the question is, can these issues be dealt with in a way that can nonetheless generate meaningful data?

The answer: sort of. Limitations on representativeness can be addressed using various approaches. There is no one-size-fits-all solution, but depending on the available data and the specific labor market, statistically weighting the data according to the industry structure derived from labor force surveys could be promising; as could comparing findings from several data sources to perform robustness checks, or simply focusing on those segments of the market with less problematic coverage bias. [7]

Deduplication issues can be solved technically to a certain extent, and there is a lot of ongoing research in this area. Essentially, most methods entail matching common fields, comparing text content and then calculating a similarity metric to determine the likelihood that two job postings are duplicates. Some job search aggregators also attempt to remove duplicates themselves – with variable success. Identifying duplicates is fairly straightforward when OJAs contain backlinks to an original ad as these links will be identical. On the other hand, job ads that have been posted on multiple job boards pose more of a challenge. Thus, ideally, multiple robust quality assurance checks should be put in place, such as manual validation over smaller data sets.

Seriously underestimated: the challenge of skills extraction

The third challenge, the level of detail, seems to be the most underestimated. OJA from the US are typically much more detailed than elsewhere. A lot of information is set out explicitly that is only implicitly available in OJA data from the UK and other countries (e.g., covered by training requirements or work experience) – or not given at all. But even within the US, this can vary greatly.

 

 

Clearly, even if we can resolve the issues concerning representativeness and duplicates, simply recording the explicit data will still result in highly unreliable nowcasts or forecasts. Instead, both the explicit and implicit data need to be extracted – together with their context. To reduce the distortions in the collected data, we need to map them accurately and semantically. This can be done with an extensive knowledge representation that includes not only skills or jobs but also education, work experiences, certifications, and more, as well as required levels and the complex relations between the various entities. In this way, we can capture more implicit skills hidden in stipulations about education, qualifications, and experience. In addition, the higher granularity of OJA data is only truly useful if the extracted skills are not clustered or generalized too much in subsequent processing, e.g., into terms like “project management”, “digital skills” or “healthcare” (see our previous post), due to working with overly simplified classifications or taxonomies instead of leveraging comprehensive ontologies with a high level of detail.

And then of course, there is the question of how to analyze the data. We will delve deeper into this in the next post, but for now, this much can be said: Even if we are able to set up the perfect system for extracting all relevant data from OJAs (and candidate profiles for that matter), we are still faced with the challenge of interpreting results – or even just asking the right questions. When it comes to labor market analyses, nowcasting and forecasting, e.g., of skills demand, combining OJA data with external data such as from surveys by NSI promises more robust results as the OJA data can be cross-checked and thus better calibrated, weighted and stratified. However, relevant and timely external data is extremely rare. And we might possibly be facing another issue. It is much easier and cheaper to up- or reskill jobseekers with, say, an online SEO course than with vocational or technical training in MIG/MAG welding. So maybe, just maybe, some of us are not that interested in the true skills demand…

 

[1] According to the 2020 Manpower Group survey, IT positions are high on the list of hardest-to-fill positions in the US, but not everywhere else. In some countries, including developed ones such as UK and Switzerland, IT professionals are not on the top 10 list at all.
[2] https://ec.europa.eu/eurostat/cros/sites/crosportal/files/SGA2_WP1_Deliverable_2_2_main_report_with_annexes_final.pdf
[3] The feasibility of using big data in anticipating and matching skills needs, Section 1.1, ILO, 2020 https://www.ilo.org/wcmsp5/groups/public/—ed_emp/—emp_ent/documents/publication/wcms_759330.pdf
[4] The ESSnet Big Data project also investigated coverage, for the detailed results see Annexes C and G in the 2018 report.
[5] https://ec.europa.eu/eurostat/cros/content/WP1_Sprint_2016_07_28-29_Virtual_Notes_en
[6] https://ec.europa.eu/eurostat/cros/sites/crosportal/files/WP1_Deliverable_1.3_Final_technical_report.pdf
[7] See for example Kureková et al.: Using online vacancies and web surveys to analyse the labour market: a methodological inquiry, IZA Journal of Labor Economics, 2015, https://izajole.springeropen.com/track/pdf/10.1186/s40172-015-0034-4.pdf