An Empirical Job Matching Model based on Expert Human Knowledge: A Mixed-Methods Approach

Figures & data

Figure 1. General methodological diagram.

The flowchart is divided into three main sections, each represented by a different background color. CV Preference of Human Experts (light blue background), Human Level Performance Determination (light green background) and Model for Job-Candidates Matching (light red background).

Table 1. Distribution of respondents working in Mexico according to data from the Stack Overflow 2023 Developer Survey in segmentation used for maximum variation sampling.

Segmentation	Number of respondents	Segmentation	Number of respondents
(0, 10] years of experience	11	2 to 99 employees	12
(10, 20] years of experience	10	100 to 999 employees	11
More of 20 years of experience	10	1000 and more employees	6

Table 2. Number of professionals to sample according to maximum variation technique through segmentation criteria of years of experience and number of employees in their current company.

		Segmentation criterion 1
		2 to 99 employees	100 to 999 employees	1000 and more employees
Segmentation criterion 2	(0, 10] years of experience	1 individual	1 individual	1 individual
	(10, 20] years of experience	1 individual	1 individual	1 individual
	More than 20 years of experience	1 individual	1 individual	1 individual

Table 3. Number of interviewed professionals organized according to the criteria of the maximum variation sample.

		Segmentation criterion 1
		2 to 99 employees	100 to 999 employees	1000 and more employees
Segmentation criterion 2	(0, 10] years of experience	3 individuals	1 individual	1 individual
	(10, 20] years of experience	1 individual	3 individuals	3 individuals
	More than 20 years of experience	1 individual	2 individuals	1 individual

Table 4. Key names of the CVs used in the different sets of the evaluation experiment.

CV ID	Greece	Trees	Constellation	Fruits
1	Delta	Pinus	Bode	Mango
2	Kappa	Betula	Messier	Orange
3	Lambda	Sequoia	Draco	Lemon
4	Sigma	Cedrus	Lynx	Grape
5	Omega	Pyrus	Triangle	Apple
6	Tau	Eucalyptus	Cigar	Blackberry

Table 5. Description of the features that each of the CVs used in the evaluation experiment had to meet.

Greece	Trees	Constellations	Fruits
Has adequate experience for the position and enunciated education.	Has adequate experience for the position, is from the area. Enunciates education.	Not from the area but has extensive experience in a related area (e.g. data engineer applying to data science). No education.	Experience appropriate for the position and does not list education.
Experience appropriate for the position and does not list education.	Not from the right area for the position but has experience in other areas. Does not list education.	Has adequate experience for the position and is from the area. Enunciates education.	Experience appropriate for the position and enunciates education.
Adequately experienced for the position but goes into great detail in each job description. Does not state level of education.	Not from the area, but from a related field (e.g., data engineer applying to data science). Expresses education.	Recent graduate with only projects. Enunciates education.	Adequate experience for the position, but the construction of the appropriate profile has occurred throughout his/her work experience (never a job explicitly as the position to which he/she is applying). Does not state education.
Overqualified for the position. Does not state education.	From the area and overqualified for the position, does not list education.	Not from the area but has experience in scattered projects related to the job he/she is applying for. Does not list education.	Adequate experience for the position. Never changed jobs. Does not state education.
Overqualified for the position in terms of work experience and goes into great detail in each job description. It states education.	Not from the right area for the position but has experience in other areas. Enunciates education.	Recent graduate overqualified in studies with little experience. However, these experiences are in indeterminate positions (not projects). Enunciates education.	With little experience in the area of the position, and lacks a few years of experience to fill the vacancy as stated. Has a bachelor’s degree, but in an area not related to the position. Has a master’s degree in a field related to the to the position.
Lacks years of experience for the position. Education provided.	From the area for the position and adequate experience. Does not state education.	Overqualified in studies with a lot of experience but only in short-term projects. Enunciates education.	Has a previous work that qualifies for the position but he/she is lacking a few years of experience to reach the vacancy. Enunciates education.

Figure 2. Baseline model architecture. The $v_i$ represents embedding vectors. Possible values of $\mathit{i}$ are $\mathit{jp}$ (job posting), and $\mathit{cv}$ (curriculum vitae). The acronym UE stands for “Universal encoder”.

Read the detailed description of this figure

The diagram illustrates the baseline model architecture for matching job postings with CVs using sentence embedding models and cosine similarity. Input Elements: Job Posting: The process starts with a job posting (blue box) which is processed by a Universal Encoder (black box). Sentence Embedding Models: Both the job posting and CV are converted into sentence embeddings by the Sentence Embedding Model (green boxes). Embedding Vectors: The output of the sentence embedding models are two vectors: embedding vector for the job posting and embedding vector for the CV. Cosine Similarity: These vectors are then compared using cosine similarity (blue box), which measures the similarity between the job posting and the CV. Threshold Evaluation: The cosine similarity score is evaluated against a threshold (blue box) to determine the suitability of the candidate. Classification: The final step is classification (green box) where candidates are labeled as suitable (1) or unsuitable (0) based on the threshold evaluation. Arrows indicate the flow of information through the various steps of the model architecture.

Figure 3. Experimental model architecture. The $v_i$ represents embedding vectors. Possible values of $\mathit{i}$ are $\mathit{jp}$ (job posting), and $\mathit{cv}$ (curriculum vitae). The acronym UE stands for “Universal encoder”.

The experimental model considers seniority requirements, semantic evaluation, and Choquet integral computation.

Figure 4. Siamese network architecture used for training the sentence embedding model. The $v_i$ represents embedding vectors. Possible values of $\mathit{i}$ are $\mathit{a}$ (the job anchor), $\mathit{p}$ (the positive example), and $\mathit{n}$ (the negative example). The acronym UE stands for “Universal encoder”.

Figure 5. Sentence embedding model architecture.

Figure 6. t-SNE 2-dimensionality reduction of the test set’s sentence embeddings with CVs and job descriptions hue by occupational areas.

Figure 7. Ranking data’s 2-dimensional singular value decomposition for every set evaluation.

Table 6. Descriptive statistics for analysis of ranking evaluation data. On the top: pair CV frequencies used on the different evaluation sets. On the bottom: marginal frequency distribution for each CV used on the different evaluation sets (RP stands for “Relative Position”).

		Greece						Trees						Constellations						Fruits
CV	CV	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]
[1,]		0	5	5	6	3	7	0	7	6	5	7	2	0	3	1	4	5	3	0	0	2	1	5	5
[2,]		2	0	3	6	3	6	0	0	2	2	3	0	4	0	3	3	6	3	7	0	7	5	7	7
[3,]		2	4	0	7	3	7	1	5	0	2	6	1	6	4	0	5	4	3	5	0	0	1	7	7
[4,]		1	1	0	0	0	5	2	5	5	0	6	1	3	4	2	0	4	2	6	2	6	0	6	6
[5,]		4	4	4	7	0	5	0	4	1	1	0	0	2	1	3	3	0	0	2	0	0	1	0	1
[6,]		0	1	0	2	2	0	5	7	6	6	7	0	4	4	4	5	7	0	2	0	0	1	6	0
		Greece						Trees						Constellations						Fruits
CV	RP	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]	[,1]	[,2]	[,3]	[,4]	[,5]	[,6]
[1,]		3	2	0	1	1	0	1	4	2	0	0	0	1	1	1	1	2	1	0	0	2	3	1	1
[2,]		0	2	3	1	1	0	0	0	1	1	2	3	0	3	1	1	2	0	5	2	0	0	0	0
[3,]		1	2	2	2	0	0	1	0	0	4	2	0	2	1	2	1	0	1	0	1	4	2	0	0
[4,]		0	0	0	2	3	2	1	1	2	2	0	1	1	1	1	1	1	2	2	4	0	0	0	1
[5,]		3	1	1	0	2	0	0	0	1	0	3	3	0	1	1	0	2	3	0	0	1	0	1	5
[6,]		0	0	1	1	0	5	4	2	1	0	0	0	3	0	1	3	0	0	0	0	0	2	5	0

Table 7. Qualities of the factor space $\mathit{X}$ and the reason behind its inclusion.

Factor	Justification
Sufficient years of experience in roles related to the position	This factor was highlighted during the interviews. In the evaluation experiment, its significance is positively reflected in all evaluation sets on CVs ranked in top positions. See the pair frequencies table of the Greece set (CVs 1 and 5), Constellations set (CVs 3 and 6), Fruits set (CVs 2 and 4), and the marginal frequency distribution table of the Trees set (CV 6).
Include descriptions of roles unrelated to the position	The evaluation experiment revealed a negative impact of this factor on CVs ranked lower in evaluation sets. See the pair frequencies table of the Trees set (CVs 2 and 5), Constellations set (CVs 3 and 6), and Fruits set (CVs 2 and 4).
Has undergraduate studies	The evaluation experiment showed the influence of undergraduate studies on the CV. See the marginal frequency distribution table of the Constellations set (CV 3) and the Fruits set (CV 2).
Has postgraduate studies	The evaluation experiment indicated the influence of postgraduate studies on the CV, especially in cases where the candidate has limited experience. See the pair frequencies table of the Constellations set (CVs 5 and 6).
Is overqualified	The evaluation experiment demonstrated the impact of this factor in the assessment of CV 4 from the Greece set. See the marginal frequency distribution table of the Greece set (CV 4).

Figure 8. Mean precision and mean recall for the CVs classification as suitable or unsuitable, using baseline and experimental models, for every set evaluation.

Mean precision and mean recall for the CVs classification for every set evaluation.

Data Availability Statement

The data that support the findings of this study are openly available in figshare at https://doi.org/10.6084/m9.figshare.25127627 and https://doi.org/10.6084/m9.figshare.25127564.