logo

To cite this report, please use the following:
Skjervheim, Ø., Bjørnebekk, O., Wettergreen, J., Grendal, O., & Stokke, G. (2023). Norwegian Citizen Panel methodology report, wave 28 [Produced by Ideas2evidence].

Background

This report describes the procedures of data collection in the 28th wave of The Norwegian Citizen Panel. Technical aspects of data collection are discussed, along with the representativity of the panel, and how survey weights are calculated.

The Norwegian Citizen Panel (NCP) is one of the main components of Digital Social Science Core Facility (DIGSSCORE) at the University of Bergen. NCP is as a collaboration between several departments at the Faculty of Social Sciences at the University of Bergen and NORCE.

ideas2evidence is responsible for the panel recruitment, the administration of the panel, and the technical solutions regarding data collection and computing.

Technical Aspects of the Survey

Software

The surveys are administrated through the web-based survey software Confirmit. Confirmit is a “Software-as-a-Service” solution, where all software runs on Confirmit’s continuously monitored server park, and where survey respondents and developers interact with the system through various web-based interfaces. This software provides very high data security and operational stability. The security measures are the most stringent in the industry, and Confirmit guarantees 99.7 percent uptime. ideas2evidence does the programming of the survey in Confirmit on behalf of The Norwegian Citizen Panel.

Pilot, Soft Launch, and Distribution

The survey went through small-N and large-N pilot testing before data collection. In addition, the survey was tested extensively during the development phase by ideas2evidence and the researchers involved in the project.

The pilot testing was regarded as successful, and no major technical revisions were deemed necessary.

The field period started by inviting a random sample of high participation respondents (soft launch). This was done in order to minimize the consequences if the questionnaire contained technical errors. No such errors were located/reported, and remaining panel members was therefore invited shortly.

Randomisation Procedures

Each wave of NCP has an extensive use of randomisation procedures. The context of each randomisation procedure may vary1, but they all share some common characteristics that will be described below.

All randomisation procedures are executed live in the questionnaire. This means that the randomisation takes place while the respondent is in the questionnaire, as opposed to pre-defined randomisations that are uploaded to the questionnaire. All randomisations are independent from another, unless the documentation states otherwise.

Randomisation functions are written in JavaScript. Math.random()2 is widely used in combination with Math.Floor()3. These functions are deployed to achieve:

  • Randomly select one value from a vector
  • Randomly shuffle the contents of an array

The first procedure is typically used to determine a random sample of respondents to separate groups, for instance a treatment group within an experiment. As an example, consider an environment where we want to separate all respondents in two groups: group 1 and group 2. All respondents are randomly assigned the value 1 or 2, where each randomisation is independent from respondent to respondent. When N is sufficiently large, the groups will be of equal size.

Here is an example of the JavaScript code executed in Confirmit:

var form = f("x1");
if(!form.toBoolean()) { //If no previous randomisation on x1
  var precodes = x1.domainValues(); //copies the length of x1
  var randomNumber: float = Math.random()*precodes.length;
  var randomIndex: int = Math.floor(randomNumber);
  var code = precodes[randomIndex];
  form.set(code);
}

The second procedure is typically used when defining the order of an answer list as random. This can be useful when asking for the respondent’s party preference or in a list experiment. However, as a party cannot be listed twice, the procedure must take into account that the array of parties is reduced by 1 for each randomisation. Here is an example4:

function shuffle(array){
  var currentIndex = array.length, temporaryValue, randomIndex;
  
  //While there remain elements to shuffle
  while (0 != currentIndex) {
    randomIndex = Math.floor(Math.random() * currentIndex);
    currentIndex -= 1;
    
    // And swap it with the current element
    temporaryValue = array[currentIndex];
    array[currentIndex] = array[randomIndex];
    array[randomIndex] = temporaryValue;
  }
  return array;
}

Previous Waves of Recruitment

Existing panel members were recruited in wave 1, wave 3, wave 8, wave 11, wave 14, wave 16, wave 18, wave 22, and wave 25. All samples were drawn from the National Population Registry of Norway. This registry holds information on everyone born in Norway, as well as former and current inhabitants. The Norwegian Tax Administration holds the formal responsibility for the registry.

Samples consist of people over the age of 18 who were randomly drawn from the registry. The extracted information was a) last name, b) first name, c) address, d) gender, e) year of birth, and f) phone number (the latter was not included in wave 1). Samples exclude people without a permanent address in Norway.

Table 1 outlines a short summary of previous recruitment efforts. Note that there are some differences between the recruitment processes. For a detailed description of each recruitment process, please refer to the respective methodology reports.

Table 1: Summary of recruitment
Event Sample size Mode Contacts Returned letters Response rate (%)
Recruitment 1 (wave 1) 25 000 Postal 2 546 20.1 %
Recruitment 2 (wave 3) 25 000 Postal, phone/SMS 4 543 23.0 %
Recruitment 3 (wave 8) 22 000 Postal/SMS 3 479 19.4 %
Recruitment 4 (wave 11) 14 000 Postal/SMS 2 334 15.1 %
Recruitment 5 (wave 14) 14 000 Postal/SMS 2 389 15.0 %
Recruitment 6 (wave 16) 34 000 Postal/SMS 2 994 14.9 %
Recruitment 7 (wave 18) 15 000 Postal/SMS 2 381 14.0 %
Recruitment 8 (wave 22) 23 000 Postal/SMS 2 623 14.5 %
Recruitment 9 (wave 25) 18 000 Postal/SMS 2 562 13.9 %

The response rate of recruitment in wave 11 and onwards were lower than recruitment in previous waves. The most important explanation is new restrictions enforced by the Norwegian Tax Administration with regards to how many times the Citizen Panel can contact the invitees. Respondents in recruitments 4-9 were contacted twice at most. Recruitment 1 also had a maximum of two contact points, but achieved a response rate of 20 percent. One explanation for why we cannot replicate a response rate of 20 percent in recruitments 4-9 might be that NCP did a lot of promotion of the panel through media outlets leading up to and during recruitment 1. Additional promotion of the panel was carried out in relation to the Norwegian Parliamentary election the same fall. We also observe a slow, declining, recruitment rate after the fourth wave of recruitment.

Data Collection

Responses by Method of Data Collection

The survey was distributed to 26 944 panel members on the 23rd and 24th of October 2023 for the softlaunch and main launch respectively. The invitation contained information on the Norwegian Citizen Panel, unique URLs for each panel member that led to the questionnaire, and unique access code which the panel members could use to log in to the survey by accessing a link on www.uib.no/medborger.

The invitation, first reminder, and second reminder were all distributed by e-mail. The third and last reminder was, depending on whether the panel member had a registered mobile phone number or not, distributed via SMS or e-mail. Prior to wave 28, 54.3 percent of the panel members were registered with a mobile phone number.

Table 2: Responses and response rate for panel members during data collection
Event Response Cumulative responses Response rate Cumulative response rate
Invitation (23rd/24th of October) 5 292 5 292 35.1 % 35.1 %
First reminder (30th of October) 2 329 7 621 15.5 % 50.6 %
Second reminder (3rd of November) 1 376 8 997 9.1 % 59.7 %
Third reminder - email (9th of November) 321 9 318 2.1 % 61.8 %
Third reminder - SMS (9th of November) 924 10 242 6.1 % 67.9 %

In total 10 242 existing panel members filled out the questionnaire. A response rate of 35.1 % was achieved between the invitation and the first reminder. Following a pattern observed in previous waves, the initial invitation produced a higher number of respondents than subsequent reminders. See table 2 for further details on number of respondents after reminders.

Using the same methodology as in previous waves for calculating response rate, respondents who have not participated in any of the last three waves are excluded. This leaves us with 15 061 eligible respondents. The overall response rate, as reported in table 2, is 67.9 %.

Approximately 1 500 of the initial invitations were reported as not delivered by Confirmit, which rounds to 5 percent. Measures are taken to ensure email deliverability, but are unable to accurately estimate how many of the delivered emails ended up as spam with the recipient.

Contact with panel members associated with an @online.no-email

During data collection in wave 27, it became evident that fewer panel members registered with an e-mail domain to @online.no responded to the survey. Telenor, the owner of the email service, made changes to their service on the 1st of June 2023, by removing a free alternative in using the service and enhancing their spam-filter. We found it likely that these changes reduced our ability to reach these panel members. Before and during wave 28 was fielded, attempts to counter these changes have been made.

A letter was distributed to the affected panel members a week before wave 28 was fielded. The letter provided information about the changes in the e-mail service, along with a link to a form where the panel memebr could change their registered e-mail address. In case the respondent wished to keep their original e-mail address, the letter also provided a step-by-step guide on how to address as a safe sender in order to avoid the spam-filter.

3 334 letters were distributed. Panel members who previously had indicated that they had moved (question fielded in wave 14 and wave 27) were exluded as address information stems from each panel members respective wave of recruitment.

424 letters were returned, in that they couldn’t be delivered to the recipient. An additional 26 panel members were deregistered for various reasons. 148 panel members accessed the form to register a new e-mail address, and 130 of those registered a new e-mail address. Of the 3 334 respondents who were contacted due to Telenor’s change in service, 940 participated in wave 28. By comparison 1 030 participated in wave 27. In wave 28 approximately 50 percent of them respondend after receiving a reminder by text message, rather than the email. Contacting the respondents by text proves to be an important tool in order to meet the issues cause by the change in Telenor’s change in email service.

Table 3: Number of respondents with an email associated with the @online.no-domain
Wave Number of respondents
Wave 24 1 683
Wave 26 1 842
Wave 27 1 030
Wave 28 940

Reactivation of inactive panel members

Most respondents fail to actively opt out of the panel before they stop answering surveys. Therefore, there are more than 10 000 inactive panel members in NCP. An inactive panel member is understood as a member who has not participated in any of the last three waves. There are many reasons why respondents become inactive. The respondent might have passed away, or they have changed their contact information and failed to update it. However, completing three surveys a year creates attrition, and inactivity can also be understood as an implicit opt-out. During wave 28, reactivating a selection of inactive panel members has been attempted.

Reactivating inactive panel members is a different approach to recruitment: it yields information on whether reactivation might cause a similar increase in active panel members to normal recruitment-processes, and how hard it is to come in contact postally with inactive panel members.

In order to test the potential in reactivation, 1 000 inactive panel members were randomly selected. None of these used the e-mail domain @online.no, or had indicated that they had moved in previous waves (r14_Pmoved and r27_Pmoved). Instead of the normal e-mail invitation, they received a postal invitation to wave 28, which was distributed at the start of the field period. Subsequent reminders were distributed by email and/or text message.

Table 4: Information on reactivation sample and for panel members who were successfully reactivated
Gender
Education
Birth year
Male Female No education/primary school Upper secondary University College/University <= 1959 1960-1989 >= 1990
Sample 47.5 % 52.5 % 10.7 % 33.8 % 45 % 20.5 % 55.8 % 22.9 %
Respondents 54 % 46 % 12.7 % 28.6 % 55.6 % 23.8 % 57.1 % 19 %

281 letters were returned as they failed to reach their intended recipient. 63 inactive panel members became respondents as a result of reactivation. Table below outlines characteristics of the sample and the reactivated respondents.

Although only 63 respondents, when comparing the characteristics of the respondents to the random sample of panel members, the distribution in age is fairly similar. It also seems that panel members with higher education is easier to reactivate, a fairly common issue in representativeness. Gender skews differently when compared to the sample. Comparisons between the two groups should be made with caution, given the relative low number of reactivated panel members.

The reactivation effort yields a reactivation rate of 8.7 % (63/(1000-281)*100). The reactivation rate is similar to the recruitment rate in the last three waves where recruitment was carried out: wave 25 8.3 %, wave 22 8.4 %, and wave 18 8.0 %, when comparing to the first point of contact, which all were letters in the aforementioned waves5. It is also similar when compared to recruitment attempts further back in time.

Response of Existing Panel Members Over Time

Comparing the number of wave 28 respondents (10 242) to the number of respondents in the previous wave 27 (9 997), gives an overall wave-to-wave retention rate of 102 percent. Figure 1 illustrates each wave of recruitment by individual lines, and shows how many respondents that are preserved for each data collection. NCP has carried out 29 waves of data collection. Depending on when the respondents were recruited, the current wave is highlighted with a red circle. For respondents recruited in wave 1, the current wave is the 29th data collection (t29). For respondents recruited in wave 22, the current wave is the seventh data collection (t7).

The wave-to-wave retention rate increases substantially after the first three waves (t1 - t3), until it stabilizes around a mean of 95 percent. Across all waves of recruitment, the current wave has a retention rate of more than 100 percent. In other words, more respondents participated in wave 28 compared to wave 27.

Figure 1: Wave-to-wave retention rate

Platforms

The questionnaire was prepared for input via smart phones, tablets, and other units capable of running web-browsers. In order to enhance the respondents’ experience, the questionnaire is responsive. Respondents on smaller devices, measured in pixels per inch (PPI), are exposed to slightly different visual representations of some questions. For instance is a question grid presented as a set of individual questions on the same page, which is different from the desktop presentation where it would be presented in a table. 50 percent of all survey respondents that opened the questionnaire used a mobile phone.

A set number of survey questions must be answered for a person to be included as a respondent. 6 percent of the mobile users did not reach this minimum requirement, compared to 10 percent for non-mobile users.

The share of mobile users is high among respondents between 18 and 45 of age. As shown in figure 2, the share of mobile users decline substantially with age.

Figure 2: Share of mobile users by gender and age

Time Usage

The average respondent used 18.1 minutes to complete the questionnaire. Measuring average time usage is a challenge, as respondents may leave the questionnaire open in order to complete the survey later. This idle time causes an artificially high average for completing the survey. The average therefore includes only the respondents that spent 60 minutes or less completing the survey.

Figure 3: Time usage distribution of survey respondents in subgroups

The questionnaire was divided into five subsets of questions in addition to questions which were exposed to all respondents. The different sets were exposed to five subgroups of respondents, determined by random for the main subsets (r28_group in the data).

Time usage across the different groups varies between 15 and 20 minutes. Respondents in group 3 spent the most time to complete the survey, while group 2 spent the least time.

Table 5: Average time usage in minutes for each subset
Type All Group 1 Group 2 Group 3 Group 4 Group 5
All users 18.1 19.3 16.2 20.1 17.3 17.7
Non-mobile users 19 20.1 17.1 21.2 18.4 18.5
Mobile users 17.1 18.5 15.2 18.9 16.2 16.9

Representativity

In this section, we describe the representativity of the panel as a whole. First, we will discuss factors explaining representativity. Thereafter we apply demographic variables to present data on representativity by different strata. The data on representativity is the foundation for the section on weighting.

Factors Explaining Representativity

There are two main points that can serve as explanations to non-response and lack of representativity when recruiting and maintaining panel members:

  • access to and familiarity with the internet (given that a web-based questionnaire was the only response mode made available)
  • the motivation and interest of the respondents

The first challenge is strongly related to the age composition of the survey respondents. Although Norway has a very high computer and internet density, the probability of having an e-mail address, and the skills required to access and fill in an online questionnaire, normally decrease with increasing age. The second challenge, motivation and interest, is often explained by the respondents’ level of education. In addition to age and education, we added the variables of geography and gender in order to test the representativity of the survey respondents. The variables have the following categories:

  • Age: 18-29 years, 30-59 years, 60 and above.
  • Highest completed education: no education/elementary school, upper secondary, university/university college.
  • Geography: Oslo, Eastern Norway, Southern Norway, Western Norway, Trøndelag, Northern Norway.

Please note that starting wave twenty-one, the former county of Akershus is part of Eastern Norway, rather than being part of the traditional Akershus/Oslo stratum. This has implications for weighting and representativity analyses, as discussed below.

Representativity of the Norwegian Citizen Panel

The sampling frame of the survey equals to the Norwegian population above the age of 18, comprising a population of approximately 4.3 million individuals. Earlier reports have documented a systematic underrepresentation of respondents belonging to the two lowest educational groups, independent of gender and age. The underrepresentation is particularly strong for young men. As expected, individuals with education from universities or university colleges are overrepresented. All of these observations hold true for wave 28.

Table 6: Age distribution in the population and the net sample
Type 18 - 29 years 30 - 59 years 60 years and above
Population 18.8 % 49.7 % 31.5 %
Net sample 4.2 % 44 % 51.8 %

From the age distribution presented in table 4, we see that the two youngest age groups are underrepresented in the net sample of wave 28 when compared to the distribution we find in the population. Inversely, respondents age 60 years and more are overrepresented in the net sample when compared to the population.

Over time, the panel has exhibited a drift away from perfect representativity with regard to age. As figure 4 shows, the oldest respondents started out as underrepresented in wave 1, but have become increasingly overrepresented over time. The youngest respondents, on the other side, have become increasingly underrepresented. This can be explained by a difference in panel membership loyalty; younger panel members are more likely to stop responding to new NCP waves after having been an active member of the panel. The rate of misrepresentation of age groups peaked with wave 26.

Figure 4: Representativity of age groups

Table 7 breaks down population and the net sample by age and gender. This reveals a slight gender-age interaction in the panel representativity. Younger men are slightly more underrepresented than younger women, while among the oldest in the panel men are more overrepresented than women in the same age bracket.

Table 7: Combined distribution of age and gender in the population and the net sample
18 - 29 years
30 - 59 years
60 years and above
Type Men Women Men Women Men Women
Population 9.7 % 9.1 % 25.2 % 24.5 % 15.1 % 16.5 %
Net sample 1.7 % 2.5 % 21.1 % 22.9 % 27.8 % 24 %

The inclusion of educational level in table 8 reveals a systematic underrepresentation of respondents with little or no education, independent of age and gender. The underrepresentation is present inn all age brackets, but is especially strong for young respondents.

Table 8: Combined distribution of age, gender and education in the population and the net sample
Population
Net sample
Men Women Men Women
18 - 29 years No education/elementary school 3.8 % 2.9 % 0.1 % 0.1 %
Upper secondary education 4 % 3.2 % 0.7 % 0.9 %
University/university college 1.9 % 3.1 % 0.8 % 1.4 %
30 - 59 years No education/elementary school 5.1 % 4 % 0.5 % 0.4 %
Upper secondary education 10.8 % 7.6 % 6.6 % 4.7 %
University/university college 9.3 % 12.8 % 14.1 % 17.8 %
60 years and above No education/elementary school 3.3 % 4.5 % 1.8 % 1.3 %
Upper secondary education 7.5 % 7.7 % 9.4 % 7.3 %
University/university college 4.2 % 4.2 % 16.7 % 15.2 %

Respondents who have completed upper secondary education as their highest completed level of education are underrepresented in all groups, except for men with upper secondary education aged 60 years or above. Those who have university or university college education are clearly overrepresented in the two oldest age brackets, irrespective of gender.

Figure 5: Representativity of education groups

Figure 5 illustrates the representation of education groups since wave one. The general trend is that the highly educated are overrepresented compared to those with less or no education. Except for slight improvements in representativity of the education groups when new respondents are recruited (wave 1, 3, 8, 11, 14, 16, 18, 22 and 25), the overall pattern has remained stable throughout all waves.

We observe that the representation of panel members living in Trøndelag, and Southern Norway are nearly on level with the population, while respondents from Northern Norway are underrepresented. Respondents from Eastern Norway, Western Norway, and Oslo are overrepresented.

Table 9: Combined distribution of age, gender and geography in the population and the net sample
Population
Net sample
Men Women Total Men Women Total
Oslo 18-29 years 1.3% 1.5% 2.8% 0.3 % 0.6 % 0.9 %
30-59 years 3.6% 3.5% 7.1% 3.8 % 4.2 % 8 %
60 years and above 1.4% 1.6% 3% 3.4 % 3.4 % 6.8 %
Total 6.3% 6.6% 12.9% 7.5 % 8.2 % 15.7 %
Eastern Norway 18-29 years 3.4% 3.1% 6.4% 0.4 % 0.8 % 1.1 %
30-59 years 9.4% 9.3% 18.7% 6.8 % 7.8 % 14.5 %
60 years and above 6.1% 6.8% 12.9% 11.1 % 9.2 % 20.3 %
Total 18.9% 19.2% 38% 18.2 % 17.7 % 36 %
Southern Norway 18-29 years 0.6% 0.5% 1.1% 0.1 % 0.1 % 0.2 %
30-59 years 1.4% 1.4% 2.8% 1 % 1.2 % 2.2 %
60 years and above 0.9% 1% 1.8% 1.3 % 1.2 % 2.5 %
Total 2.9% 2.9% 5.7% 2.4 % 2.5 % 4.9 %
Western Norway 18-29 years 2.6% 2.4% 4.9% 0.6 % 0.5 % 1.1 %
30-59 years 6.4% 6.1% 12.6% 6.3 % 6.2 % 12.5 %
60 years and above 3.8% 4.1% 7.9% 7.8 % 6.6 % 14.4 %
Total 12.8% 12.6% 25.4% 14.7 % 13.3 % 28 %
Trøndelag 18-29 years 0.9% 0.9% 1.8% 0.2 % 0.4 % 0.6 %
30-59 years 2.2% 2.1% 4.2% 1.7 % 1.8 % 3.5 %
60 years and above 1.3% 1.4% 2.8% 2.2 % 1.8 % 4 %
Total 4.4% 4.4% 8.8% 4 % 4 % 8 %
Northern Norway 18-29 years 0.9% 0.8% 1.7% 0.1 % 0.1 % 0.3 %
30-59 years 2.2% 2.1% 4.3% 1.5 % 1.8 % 3.3 %
60 years and above 1.5% 1.6% 3.1% 2.2 % 1.7 % 3.9 %
Total 4.6% 4.5% 9.1% 3.8 % 3.7 % 7.5 %

Respondents aged 60 years and above are overrepresented in all parts of the country, which is especially so for Oslo, Eastern Norway, and Western Norway. Conversely, respondents aged 18-29 years are underrepresented in all regions.

Figure 6: Representativity of regions

For wave twenty-one, population data stratified on the new regions was available for the first time since the regional reform of 2020. While this data eliminates some small uncertainty in the representativity analyses6, it also introduces a break in time series for Oslo (previously including Akershus) and Eastern Norway (now including Akershus). Eastern Norway now makes up almost one fourth of the population, the diversity of which we are no longer able to account for in full in the respondents’ weights. Compared to age and education, geography does, however, not seem to be a strong determinant of survey participation. Apart from effects from the regional reform, the geographic representativity is more or less stable over time.

Weighting

A weight has been calculated to compensate for observed biases. The weight is equal to the relation between a given strata in the population and the total population, divided by the relation between a given strata in the net sample and the total net sample7. This procedure returns values around 1, and always more than 0. Respondents belonging to a stratum that is underrepresented will receive a weight above 1 and respondents belonging to an overrepresented stratum will receive a weight below 1. We have listed the weights of the different strata in the table in the appendix.

When calculating the weight, information regarding the respondents’ geographical location, gender, and age is based on registry data. Information on these variables was included in the sample file received from the Norwegian National Population Registry upon recruitment. Information on the level of education is collected from NCP surveys. 1 percent of the 28th wave net sample have not answered the question about level of education.

The following demographic variables are applied in weight4:

When applied, the weight will provide a weighted N equal to the number of cases in the dataset. In other words, the weight is calculated using the whole dataset. NCP has an extensive use of (randomized) sub-groups, which might alter the demographic profile of the sub-group compared to the whole dataset.

Consequently, the weight might be less precise for some sub-groups. Note that the dataset is provided with necessary information8 to calculate custom weights if needed, following the procedure described above.

Table 10 shows the effects of weight 4 on the distribution of self-reported level of education in the net sample. As we can observe, the weight gives the sample a distribution closer to the population. It is, however, important to stress that the unweighted distribution is not ideal, with a clear underrepresentation of people with low levels of education.

Table 10: Effect of weight4 on self-reported level of education
Sample - standard Sample - weighted Population Difference between sample and population Difference between weighted sample and population
Upper secondary school and lower 33.9 % 62.5 % 64.4 % -30.5 -1.9
University/university college 66.1 % 37.5 % 35.6 % 30.5 1.9

Appendix

Table 11: Weight4 applied to different strata
Age Education Men Women Age Education Men Women
Oslo 18-29 Upper secondary and lower 5 4.8 Western Norway 18-29 Upper secondary and lower 5 5
University/university college 1.9 1.8 University/university college 2.3 3.6
30-59 Upper secondary and lower 2.7 2.3 30-59 Upper secondary and lower 1.9 2.1
University/university college 0.7 0.7 University/university college 0.6 0.7
60+ Upper secondary and lower 1.1 1.1 60+ Upper secondary and lower 0.9 1.3
University/university college 0.3 0.3 University/university college 0.2 0.3
Eastern Norway 18-29 Upper secondary and lower 5 5 Trøndelag 18-29 Upper secondary and lower 5 4.3
University/university college 3.3 2 University/university college 2.7 1.6
30-59 Upper secondary and lower 2.6 2.5 30-59 Upper secondary and lower 3 3.2
University/university college 0.8 0.8 University/university college 0.7 0.8
60+ Upper secondary and lower 1 1.5 60+ Upper secondary and lower 1.1 1.9
University/university college 0.3 0.3 University/university college 0.3 0.3
Southern Norway 18-29 Upper secondary and lower 5 5 Northern Norway 18-29 Upper secondary and lower 5 5
University/university college 3.6 2.4 University/university college 2.8 4.2
30-59 Upper secondary and lower 2.1 2.2 30-59 Upper secondary and lower 2.7 2.2
University/university college 0.8 0.8 University/university college 0.8 0.9
60+ Upper secondary and lower 1.5 2.1 60+ Upper secondary and lower 1.2 2.1
University/university college 0.3 0.3 University/university college 0.3 0.3

  1. Some examples: sorting respondents in different thematic subsets, randomly allocate treatment value in experiments, randomize order of an answer list/array, order a sequence of questions by random, ask a given question to a subset of the respondents.↩︎

  2. Please see following resource: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random↩︎

  3. Please see following resource: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/floor↩︎

  4. Collected from Mike Bostocks visualisation: https://bost.ocks.org/mike/shuffle/↩︎

  5. See the documentation report for each wave for more information.↩︎

  6. Note that Oslo (including Akershus), and Eastern Norway diverge in wave eighteen, due to the regional reform implemented 1st of January 2020.↩︎

  7. The applied formula for weight wi for element i, in strata h is: \[wi=\frac{N_h/N}{n_h/n}\]↩︎

  8. See columns r28_Weight4_stratapop and r28_generic_stratapop↩︎