Development and validation of the Italian digit-innoise test

Cover figure: Average speech recognition probabilities for each individual digit among the 35 subjects with bilateral pure-tone averages (PTA) ≤ 15 dB (group c), as a function of signalto- noise ratio (SNR).

Objective. This research aims to validate the digits-in-noise (DIN) test for the Italian language and develop a version capable of independently assessing both ears while maintaining acceptable administration times.
Methods. Individual digits from 0 to 9 in Italian were recorded and adjusted to equalise recognition probabilities. An iOS application (APP) was developed for the independent ear test using triplets in noise. The application incorporates a new proprietary adaptive procedure developed by Amplifon to minimise the number of steps required to determine the Speech Reception Threshold (SRT). Thirty-nine subjects were recruited for equalisation of digits, 45 normal-hearing and 62 with various degrees of hearing loss for normative-data assessment.
Results. The results demonstrate the ability to determine a threshold value for normal hearing consistent with the existing literature and identify threshold values corresponding to the main World Health Organization hearing loss categories.
Conclusions. A DIN test for the Italian language has been developed and validated to evaluate the SRT of each ear individually. The adaptive algorithm optimises the necessary steps while maintaining acceptable test duration for both ears. Users can autonomously conduct the test using a standard personal iOS device (tablet or smartphone).

Introduction

One of the primary challenges for individuals with hearing impairment is the difficulty in understanding speech in the presence of background noise. Generally, tonal audiometry thresholds provide limited insights into speech comprehension in noisy environments ^1,2. Thus, specific speech comprehension tests in noisy environments are necessary. These tests aim to determine the signal-to-noise ratio (SNR) at which 50% of speech content is recognised, known as the Speech Reception Threshold (SRT). Clearly, it is essential to develop and validate such tests for each language.

Currently, several types of speech comprehension tests in noisy environments exist for the Italian language (e.g.: SPIN, Matrix), primarily utilised in clinical practice. However, there is a lack of available tests based on linguistic material that can be quickly self-administered, for instance, via personal devices. Such tests should be usable by individuals with hearing loss (HL), hearing aids, and cochlear implants, independent of linguistic proficiency.

To address this need, many other countries have developed and utilised tests in their respective languages based on numerical comprehension in noisy environments for evaluating SRT: the digits-in-noise (DIN) test. Numerical material represents one of the most familiar phonetic categories understood by both young and elderly individuals. Thus, it was deemed necessary to develop and validate a numerical auditory test for the Italian language.

The use of numbers as phonetic research material dates back to the 1950s when Milner ³ utilised this technique to explore contextual effects on speech intelligibility. Erber ⁴ developed an audiometric test using numbers for children aged 3 to 8 with significant hearing impairment. Wilson ^5-7 and colleagues conducted numerous studies examining the flexibility of using numbers organised in triplets to create speech-in-noise tests. They also investigated the feasibility of using numbers organised in pairs or triplets with babble noise to assess speech comprehension with a distractor element. Building upon these findings, Van Wierigen ⁸ developed a test with numbers, with and without distractor elements ⁹, for patients using a cochlear implant or with severe HL. In another study, McArdle and Wilson ¹⁰ examined the recognition of the acoustic, phonetic, and lexical aspects of these number recordings and found that only 3% of the variance was due to lexical elements. Through further studies, Smits ^1,2,11,12 developed a screening test to be conducted via telephone ¹³, introducing the DIN test for the first time as it is now conceived. The test was launched as the National Hearing Test in the Netherlands in 2003, achieving considerable success due to its simplicity and reliability, thus becoming the model for future DIN tests. The adaptive procedure involves increasing or decreasing fixed-size noise steps after each correct or incorrect response.

Using this test as a reference, other countries including the Netherlands ¹¹, Germany ¹⁴, Poland ¹⁵, France ¹⁶, United States of America ¹⁷, Turkey ¹⁸, China (Mandarin) ¹⁹ and Korea ²⁰ have developed the DIN test in their respective languages. Thanks to its speed and simplicity, enabling administration without an audiometer, the test has provided many more people with the opportunity to undergo a clinically valid and reliable examination. In 2016, the University of Pretoria ²¹ validated the test for South Africa and developed an Android application ²², allowing anyone with a smartphone to perform the test as a self-assessment tool for hearing status. During the research, it was also demonstrated that the test is independent of the type of earphone used.

Given the ongoing rise in hearing impairments, especially among young individuals, coupled with reduced awareness of the problem and lower prevalence of hearing aids compared to other countries, there is a growing need in Italy for a hearing test to assess the SRT in a simple and direct manner, to extend self-assessment screening and use in audiological practice.

The main objective of this research is to validate the DIN test for the Italian language, thereby providing a simple and fast test for our language, potentially deliverable via personal devices, as is the case in several countries worldwide. The availability of this test in the Italian language will provide an agile and rapid diagnostic tool suitable for broad-spectrum screening activities, particularly useful in contexts where there is a high percentage of non-native Italian speakers. From a procedural standpoint, several adaptations of the DIN test exist compared to the original version by Smits et al. ¹. Some assess each ear separately ^1,16,17, while others use identical binaural stimuli ²¹, primarily to reduce test duration. However, the diotic test fails to detect unilateral neurosensorial HL due to the dominance of the better ear ²³. In the present study, a monaural version of the test was developed, capable of individually assessing each ear, as the longer administration times compared to the binaural version were reduced using an adaptive algorithm that minimises the necessary steps for SRT evaluation.

Materials and methods

The research was divided into three phases (Fig. 1) following a protocol widely used in the literature ²¹ and according to the recommendations of the International Collegium of Rehabilitative Audiology ²⁴.

Phase 1: recording and processing of vocal material

This phase involved the recording of individual numbers by professional speakers, selection of the best recordings based on criteria expressed in the literature, digital processing of samples to ensure consistent sound levels, and generation of vocal masking noise from the recorded numbers.

For the studio recording session, six lists of 10 numbers were created as reading material, with the numbers presented in random order so that each number appears six times. Additionally, each digit was introduced by the carrier-phrase “il numero” to facilitate intonation and correct pronunciation. The lists were read by two professional speakers in two separate recording sessions, using the same equipment and environment.

This phase was conducted at the AltreFrequenze studio in Brescia, using a Neumann U87 Ai microphone and a sampling frequency set to 48,000 Hz with a 16-bit resolution. Each number was obtained through cutting operations using Pro Tools software. The 120 audio files obtained were equalised to the same Root Mean Square (RMS) level before being saved in .wav format. RMS represents the effective value of sound pressure, ensuring that the audio files are at the same sound pressure level, thus achieving uniform playback.

The selection of the most suitable voice for the test was made in collaboration with a team of ten speech therapists from the Audiology and Phoniatrics Service, ENT Department, University of Modena e Reggio Emilia. They first selected the speaker with the most natural voice and intonation, and then chose the best pronunciation for each number, based on the following evaluation criteria ²¹:

• naturalness;
• articulation;
• voice quality;
• intonation;
• playback speed.

The female voice with the highest average evaluation and the best pronunciation for each number by the selected speaker were chosen.

Thus, the 10 best audio files were identified to be used as phonetic material for the test.

Phase 2: equalisation of digits

This phase aimed to equalise the digits based on their recognition probability. Although the digits are presented at the same intensity level, each digit has a different probability of being recognised based on spectral content and envelope, which varies depending on the language. Equalisation of the digits was achieved by applying level corrections to individual digits, ensuring that each digit has a 50% probability of being recognised correctly at the same SNR.

Each presentation begins with 500 ms of noise, followed by the digit, and then another 500 ms of noise. Each digit is presented once in each list in random order at each SNR. The masking noise was set at 70 dB SPL. The presentation starts with the easiest SNR and progresses to more difficult ones. The noise remains fixed, and the intensity of the digit decreases. For each level and for each list, the digits are presented in random order. The list of the digits is stored in a separate file for comparisons.

Thirty-nine subjects with an average age of 21 years and a range between 18 and 35 years were selected. The audiometric test was conducted using an Inventis Modello Piccolo Speech audiometer controlled by a laptop PC, with Maestro audiometric software and soundproofed Audiocups audiometric headphones. The 39 subjects underwent tonal audiometric examination at frequencies of 125, 250, 500, 750, 1000, 1500, 2000, 3000, 4000, 6000, and 8000 Hz, with the following results:

• a) 15 subjects had all thresholds bilaterally ≤ 15 dB HL;
• b) 26 subjects had all thresholds bilaterally ≤ 20 dB HL;
• c) 35 subjects had a bilaterally Pure-Tone Average (PTA) ≤ 15 dB HL.

For the equalisation test, software was developed in the MAX MSP environment, which is a graphical development tool fotor design software dedicated to real-time audio and multimedia applications, designed and developed at IRCAM in Paris.

The noise level was calibrated so that the headphone output was at 70 dB SPL by measuring it using a B&K mod. 2250 sound level meter with B&K mod. 4152 artificial ear and 6 cc coupler, with the audio output of the PC calibrated to full scale. The professional AKG mod. k 240 studio headphones were used.

Subjects were presented with the four lists of 100 digits, always starting from the right ear and alternating between ears to present two lists to the right ear and two lists to the left ear. Subjects had to listen to each digit and enter the response on the laptop keyboard. The next digit was presented after the subject responded by entering the digit on the keyboard. If the subject was unable to identify the digit, they still had to enter a value by trying to guess the digit. The responses of each subject were recorded.

The data collected allowed for the definition of the intelligibility curve of each digit. Each digit was then increased or decreased by a level equal to the difference between its 50% recognition value and the average of the 50% recognition levels of all the digits. This allows the generation of 10 digits with equal recognition probability for use in the next phase.

Results

Each response was considered correct if the entered digit matched the reproduced one. Since the test was administered to 35 valid subjects, the same digit at the same intensity was reproduced a total of 140 times (in one test, the same audio appears 4 times). Speech intelligibility (or recognition) functions were calculated for the three classes of subjects (a, b, c) with very similar results.

Group c, composed of 35 subjects with PTA ≤ 15 dB bilaterally, was chosen as the statistically relevant sample as the average slope is approximately 24%, which is closest to the values reported in the literature (around 20%) ¹⁹.

The data in Tables I and II summarise the response trends as a value and as a percentage for each digit at the corresponding SNR for group c. The speech recognition function for each digit was determined by fitting a logistic function to the raw data using a maximum likelihood estimation procedure.

The analysis was performed using JAMOVI software by applying a binomial logistic regression with maximum likelihood estimation to the responses of the 35 subjects, one digit at a time. All results showed statistically significant outcomes (p < 0.001), as reported in Table III.

Thus, values were obtained for each digit (0-9) as follows:

• α = estimated constant coefficient (intercept);
• β = estimated variable coefficient (dB).

The average vocal recognition probabilities for each digit as a function of SNR for group c are shown in Cover figure.

The SNR corresponding to 50% recognition for each digit was determined by inserting the parameters α (intercept) and β (dB) into the inverse logistic function as shown in Table III. The mean value for all digits was found to be -12.8 dB.

The correction factor for each digit was calculated by subtracting its relative SNR from the average SNR of all digits²⁵ and will be used in the DIN test to align the correct recognition probabilities to 50% for all digits.

Phase 3: normative data assessment

The purpose of this phase was to determine the range of values of the SRT in noise, capable of discriminating results between normal-hearing subjects and those with hearing impairments, and among these, the different degrees of HL according to the categories identified by the WHO with their respective PTA intervals. To achieve this, the DIN test was administered to groups of normal-hearing and hearing-impaired subjects with different degrees of HL and compared with the results of traditional audiometry. The respective SRT values were identified for each group.

For the DIN test, lists of numbers containing unique triplets of digits, randomly selected and equalised with the previously calculated values, were used. The test utilises a fixed noise level and a variable speech level; the masking noise used is babble noise. The adaptive test procedure is as follows:

• the first triplet is presented at an intensity of 60 dB;
• the subject must enter all three digits heard on a numeric keypad;
• upon response entry, the next triplet will be presented with a higher SNR for an incorrect response or with a lower SNR for a correct response with a step of +/- 2 dB;
• a triplet is considered correct when all digits are entered correctly.

Similar to the equalisation phase, the test was administered individually to each ear of the patient. Therefore, responses were categorised based on the PTA of each individual ear. The test was administered to a total of 107 native Italian-speaking subjects, for a total of 214 ears tested, with the distribution shown in Table IV.

Normal-hearing and hearing-impaired patients were recruited from the Audiology Department of Magna Graecia University of Catanzaro. The mean age of subjects was 48 years (SD = 19 years) ranging from 19 to 77 years. Sixty subjects were females and 47 were males.

The subjects underwent the DIN test first and then traditional audiometric examination.

The subjects carried out PTA with the following frequencies 125, 250, 500, 750, 1000, 1500, 2000, 3000, 4000, 6000, and 8000 Hz.

Instruments:

• PTA:
  • audiometer: Astera 2- Otometrics;
  • audiometric earphones TDH-39P Telephonics;
• DIN test:
  • iOS app developed by Amplifon;
  • circumaural headphones radioear DD65 v2;
  • initial presentation calibrated at 60 dB;
  • iPad Air device.

To ensure the highest output accuracy, the software takes into account the Apple hardware in use and allows calibration of the headphones based on predetermined values, specifying the model of the connected audiometric headphones.

The vocal material used by the app was previously prepared and stored as uncompressed WAV files at 24-bit with a sampling frequency of 44100 Hz.

Once installed on the device (iPad or iPhone), the app does not require an internet connection during test execution.

The adaptive procedure developed by Amplifon is as follows:

• the test begins on the first ear (right) with a conditioning phase (not explicit to the subject) aimed at determining an initial SNR value from which to start collecting SNR values of various responses;
• during conditioning, dynamically generated triplets are presented in a randomised manner by the software, preventing repetitions of similar cases, and based on the correctness of responses, the speech volume is increased or decreased by 4 dB;
• the conditioning phase ends after a correct response following one or more incorrect responses, or after an incorrect response following one or more correct ones. The conditioning phase also ends if the speech volume reaches one of the limits (min: 0 dB, max: 90 dB);
• once conditioning is completed, the speech amplitude is set ±2 dB (based on the correctness of the last response) and then 8 triplets are presented, and their respective SNR values are saved;
• the speech volume is adjusted at each iteration based on the correctness of the previous recognition;
• the average SNRs are then calculated, and the process is repeated starting from conditioning the left ear;
• Amplifon’s algorithm may lead to SNR calculation with a minimum of 10 iterations (2 for conditioning and 8 for the SNR average calculation) per ear and may vary depending on the duration of conditioning.

Threshold values for normal hearing and for different categories of HL were determined through the upper 90th percentile point. Pass/refer values for each category are illustrated in Table IV. The Pearson correlation coefficient between the DIN test in Italian and PTA was r = 0.83 with p < 0.001, which is in line with the values reported in the literature ^1,17,20,22.

Discussion

A DIN test has been developed and validated for the Italian language. The application developed for tablets and smartphones offers wide usability as a screening tool for all age groups. The signal produced is broadband and is not subject to streaming limitations since it is generated by the device itself, with a bandwidth ranging from 30 to 20,000 Hz, capable of representing human voice with maximum detail, thus facilitating comprehension. The test can be performed for self-assessment and provides specific indication for each ear.

In Phases 1 and 2 of this study, the Italian-based digit hearing test was developed following procedures similar to those of DIN tests in Dutch, French, and South African languages. The average diotic SRT value to discriminate between normal-hearing subjects and those with HL was -7 dB, consistent with values obtained for Dutch, French, and German languages ranging from -6.4 to -6.9 dB.

The test was applied to both normal-hearing patients and patients with varying degrees of HL, allowing the determination of normality thresholds and the identification of thresholds corresponding to different categories of HL according to the WHO. The normative data assessment was conducted using audiometric headphones to maintain a reference value for future use.

Conclusions

A digit-based hearing test for the Italian language (I-DIN) has been developed and validated, which can be performed by users independently. Based on similar works that have previously demonstrated the validity of the test even using commercial headphones, the test can also be used with personal headphones and independently. The test also allows for the separate evaluation of both ears.

An adaptive algorithm for digit triplets was used to minimise the number of presentations, thus shortening the duration of the test for both ears to temporal dimensions similar to those of the diotic version.

Therefore, the DIN test in the Italian language can be used for screening activities. Based on these initial results and in the context of large-scale delivery using personal headphones, we plan to validate the tests on both normal-hearing and hearing-impaired subjects using commercial headphones to verify that the presence of background noise or a possible reduced bandwidth of the headphones does not affect the validity of the test for the Italian language, even though the scientific literature ²¹ has already extensively validated this aspect for other languages. The APP will also be developed for the Android platform to be usable on all types of personal devices.

Correlations with other tests (Matrix or SPIN) in Italian will also be evaluated in future research. Large-scale delivery is also planned to assess any statistical fluctuations due to age or gender and verify that supra-aural factors are uncorrelated with test results. Furthermore, a diotic version will be developed for use with both headphones and in free-field settings for hearing aid fitting activities, and version with antiphasic signals, to discriminate sensorineural HL from conductive ones. Finally, spatialised versions are under study for use with both headphones and in free-field settings to improve the test’s adherence to real listening situations.

Acknowledgements

The authors thank all the participants of this study.

Conflict of interest statement

The authors declare no conflict of interest.

Funding

This work was partially supported by Amplifon. The funder had no role in the design of the study and in the collection and analysis of the data but only in the conceptualisation, design and implementation of the proprietary adaptive algorithm for the delivery of digit triplets in noise during the normative assessment of the data and implementation of the DIN test for use by patients.

Author contributions

PV: principal investigator; PV, GC, AM, CB: conceptualisation; PV, GC, AM: methodology; AA, AM, AS: formal analysis and investigation; GP, CB: software tools design and develop; AM, PV: writing original draft preparation; AM, PV, GC, CB: writing – review and editing.

Ethical consideration

This study was approved by the Institutional Ethics Committee “Comitato Etico Sezione Area Centro – Regione Calabria” (approval protocol: 197, 16/6/2022).

The research was conducted ethically, with all study procedures being performed in accordance with the requirements of the World Medical Association’s Declaration of Helsinki.

Written informed consent was obtained from each participant/patient for study participation and data publication.

History

Received: May 8, 2024

Accepted: July 21, 2024

Figures and tables

Figure 1. Phases of the development and validation protocol of the Italian digit-in-noise test.

Table I. Results of the equalisation test, indicating the number of correct responses for each digit at the corresponding signal-to-noise ratio (SNR) for group c.

	Digits
SNR	0	1	2	3	4	5	6	7	8	9
-20	14	17	13	15	24	62	15	4	6	11
-18	9	30	20	10	43	115	12	6	3	11
-16	43	22	11	17	17	127	31	10	15	10
-14	23	11	24	22	64	139	66	37	55	22
-12	89	38	26	18	138	140	64	60	47	52
-10	118	41	53	126	139	139	129	89	135	123
-8	139	120	84	133	139	140	136	54	140	140
-6	135	126	115	139	138	140	137	137	140	140
-4	135	138	140	135	137	140	137	118	139	140
-2	140	138	138	137	140	140	140	131	140	140
Total	845	681	624	752	979	1282	867	646	820	789

Table II. Results of equalisation test, showing the percentage of correct responses for each digit at the corresponding signal-to-noise ratio (SNR) for group c.

	Digits
SNR	0	1	2	3	4	5	6	7	8	9
-20	10%	12%	9%	11%	17%	44%	11%	3%	4%	8%
-18	6%	21%	14%	7%	31%	82%	9%	4%	2%	8%
-16	31%	16%	8%	12%	12%	91%	22%	7%	11%	7%
-14	16%	8%	17%	16%	46%	99%	47%	26%	39%	16%
-12	64%	27%	19%	13%	99%	100%	46%	43%	34%	37%
-10	84%	29%	38%	90%	99%	99%	92%	64%	96%	88%
-8	99%	86%	60%	95%	99%	100%	97%	39%	100%	100%
-6	96%	90%	82%	99%	99%	100%	98%	98%	100%	100%
-4	96%	99%	100%	96%	98%	100%	98%	84%	99%	100%
-2	100%	99%	99%	98%	100%	100%	100%	94%	100%	100%
Total	60%	49%	45%	54%	70%	92%	62%	46%	59%	56%

Table III. Results of binomial logistic regression with maximum likelihood estimation to the responses of the 35 subjects with bilateral PTA ≤ 15 dB HL (group c), for each digit; SNR corresponding to 50% recognition for each digit; mean value for all digits (-12.8929 dB) and calculated correction factor for each digit.

Digits	0	1	2	3	4	5	6	7	8	9
y (%)	0.5	0.5	0.5	0.5	0.5	0.5	0.5	0.5	0.5	0.5
Alpha (intercept estimate)	6.116	3.74	3.529	5.821	7.584	13.972	6.247	3.44	8.397	7.619
Beta (dB estimate)	0.466	0.349	0.358	0.495	0.503	0.706	0.465	0.338	0.66	0.621
Ln	0	0	0	0	0	0	0	0	0	0
p	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001
dB 50%	-13.12	-10.72	-9.86	-11.76	-15.08	-19.79	-13.43	-10.18	-12.72	-12.27
Mean	-12.8929
Equalisation (dB)	0.23	-2.18	-3.04	-1.13	2.18	6.90	0.54	-2.72	-0.17	-0.62

Table IV. Threshold values for the DIN test in Italian for the different degrees of WHO hearing loss calculated with the 90th percentile.

WHO Hearing Loss Classification	Limits in dB	SNR Italian DIN test in dB	Number of ears tested	Min SNR	Max SNR
Normal hearing	< 20	-7	90	-17	-2
Mild loss	≥ 20 < 35	-2	41	-15	1
Moderate loss	≥ 35 < 50	5.8	33	-10	30
Moderate-severe loss	≥ 50 < 65	11.9	28	-7	30
Severe loss	≥ 65 < 80	27.6	13	-3	30
Profound/complete loss	≥ 80	30	9	1	30