INTRODUCTION

Rapid development of information and communication technologies in recent years have accelerated the utilization of the Internet to access health-related information owing to its advantages such as being easily accessible, anonymous and cheap. People use health-related web sites for several purposes: receiving information about diseases, prescriptions written by their physicians, or general health conditions or medications prior to the examination; seeking other medicines with similar qualities to those on their prescription; purchasing prescription or non-prescription medicines; benefiting from product discounts or interactive health-related e-mail services; and communicating with other patients (1,2). In addition, patients also use the Internet for purposes such as participation in psychosocial groups, caring for their personal needs, and sharing in decision-making with physicians (3). As a result, online health information acquisition is widely utilized by the Internet users (4,5). While it can be positive and preventive in terms of exercising, adopting healthy eating habits, complying with drug treatment, making competent health decisions (6,7); online health seeking can be problematic when it is used as a diagnostic tool for non-specialist individuals (8).

Despite providing medical information through easy-to-understand diagnosis web sites makes it possible for non-health professionals to better understand health conditions and diseases and to present logical explanations for their symptoms, Internet has the potential to increase anxiety of people with little or no medical knowledge, particularly when used as a diagnostic tool (9). Benigeri and Plue (10) stated that non-medical educated individuals might face the risk of self-diagnosis and treatment when they are exposed to complex medical terminology.

Considering the purposes served by health-related information-seeking behavior on the Internet, it is thought to be another form of assurance-seeking behavior to acquire definite, accurate, non-confusing, and confidential information related to existing health conditions (2). Assurance seeking, and anxiety are the main features of repetitive health-related information-seeking behavior on the Internet (2,11). After getting health-related information through the Internet, some people feel more competent, while others may be anxious and confused (9). Individuals may become distressed when faced with unreliable, confusing, and uncertain information (12,13). In other words, while the individual aims to gain benefits for oneself or his/her relatives, he or she also increases the possibility of getting harm by this behavior.

At this point, a new concept, cyberchondria, has emerged in the literature. This new concept was introduced by the large scale study of White and Horvitz (9). Cyberchondria is described as the feeling of intensive anxiety of unknown origin about non-specific symptoms associated with health-related information seeking on the Internet (9). Starcevic and Berle (2) defined cyberchondria as the long-term intensive or repetitive use of the Internet to reduce health-related experience of stress or anxiety (resulting in further deterioration of the existing symptoms). Nevertheless, it is noteworthy that cyberchondria defines a behavioral, emotional, and cognitive pattern, not a diagnosis yet (11). Health anxiety, which has a strong association with cyberchondria appears in this context. Defined as the experience of excessive distress or anxiety that is related to the health of the individual as a result of misinterpretation of his/her somatic sensations and consequent belief of having a serious disease, health anxiety can occur at varying severity and is regarded as hypochondria when it becomes severe (14,15). On the other hand, cyberchondria has a multi-dimensional structure including anxiety and compulsion; and may lead to psychological distress, concern, and unnecessary medical expenses (11). Despite being perceived as similar, several studies suggested cyberchondria to differ from health anxiety and obsessive-compulsive symptoms (16). Main point of understanding cyberchondria is the increasing anxiety experienced in the progression from non-specific symptoms to serious diseases during health-related search on the Internet (17).

It has been stressed that there was a positive correlation between health anxiety and cyberchondria covering health-related information-seeking behavior on the Internet (18-21), and that individuals with higher scores of health anxiety spent more time on the Internet (22), and that these people experienced more anxiety and distress based on the information they got after health-related information-seeking on the Internet (22,23). A study reported one in every five subjects with low levels of health anxiety to have increased level of anxiety after a health-related Internet search (24). Searching for information about health from the Internet seems to cause a vicious circle by increasing anxiety and uncertainty in individuals with health anxiety (23). Several studies reported that concern and anxiety originated by health-related information seeking on the Internet may lead to deterioration in the functioning (9,24,25) and that elevated cyberchondria scores were associated with a decline in psychological well-being (17).

Some cyberchondria studies focus on the relationship with health services. Health-related Internet search often causes acquisition of information about local health services (26) and may lead to deterioration in the physician-patient relationship (27,28). This may cause further cost increment in health care services (e.g. doctor shopping) (11). Considering all these, it becomes once more important to define, understand, conceptualize, and measure the cyberchondria, which is closely related to both the subjective health status of the individual and the social health expenditures.

Several scales developed to measure cyberchondria have been reported in the literature. The scale developed by Fergus (29) proposed a five-factor structure, but it has been reported that subdomains other than mistrust to healthcare professionals were more likely to better explain the nature of cyberchondria and that cyberchondria exhibited a stronger and more significant association with health anxiety compared to that with obsessive-compulsive symptoms. Psychometric features of another scale developed by McElroy and Shevlin (11) were investigated in university students. It was not developed to diagnose cyberchondria in a categoric manner, but rather dimensionally to obtain information about the level of distress. This scale aims to investigate that how the individuals perform their online health seeking, the level of distress they regard online health seeking, and that how it affects their activity on and outside the Internet (11). After performing a validity and reliability study in adult sample, Cyberchondria Severity Scale (CSS) has been introduced to the Turkish literature (30). It was reported that the scale included subdimensions of compulsion, excessive anxiety, excessiveness, reassurance, and mistrust of medical professional and propositions of these subdimensions were in line with the original form except that items regarding reassurance and extravagancy subdimensions appeared to overlap. The psychometric properties of the short form of the scale (CSS-15) were also determined (18,31).

The scale developed by McElroy and Shevlin (11) was adapted and used to measure cyberchondria in Turkey. It rather focuses on the symptoms cyberchondria and getting information about the negative effects of these symptoms yet provides no information about the web page characteristics and/or health-related information-seeking characteristics that decrease/increase symptom-related anxiety, potential impacts of cyberchondria on patient-physician interaction, and no data on hypochondria. These limitations of the scale led us to plan to develop a novel scale. For this purpose, an extensive literature search was conducted; and considering the multi-dimensional structure of the cyberchondria, a scale described in following sections was designed. The aim of the study was to develop a multi-factorial and self-reported Cyberchondria Scale (CS) based on our cultural characteristics and to determine its psychometric properties.

METHOD

Sample-I

A total of 250 subjects who are aged between 18-65 years (mean=36.57, SD=14.27), live in Ankara and can use the Internet constituted the study sample. The gender was evenly distributed, where 49.6% were women. The level of education status consisted of primary education in 13.6%, high school in 27.6%, and university or higher in 58.8% of the participants. The study sample was created by convenience sampling technique (32). The individuals who responded “yes” to the question of “is there a psychiatric diagnosis you have got in the last six months?” in the demographic data form were not included to the sample.

Sample-II

The study sample consisted of 360 subjects who are aged between 18-65 years (mean=36.51, SD=13.63), live in Ankara, can use the Internet, and have no psychiatric diagnosis. Gender distribution showed 61.1% (n=220) women and 38.3% (n=138) men. The level of education status consisted of primary school in 10.3%, high school in 28.6%, and university or higher in 60.5% of the participants. Gender, age, and level of education were not specified by two, eight, and two participants, respectively. The study sample was created by convenience sampling technique. Individuals who declared to have a history of a psychiatric diagnosis established within the last six months were not included to the study sample. This sample was only used for Confirmatory Factor Analysis (CFA).

Measures

Apart from CS, the scales used in this study included the Internet Addiction Scale (IAS), Brief Symptom Inventory (BSI), and the Health Anxiety Inventory (HAI) which had been proven to be valid and reliable.

Demographic Data Form: It is a form prepared by the investigators to acquire information about the participants’ demographic data, such as age, gender and education, and whether they have received any psychiatric diagnosis within the last six months.

Cyberchondria Scale (CS): In the first stage of the development, the literature analyzing the concept of cyberchondria and related variables to form the statements to be included in the scale has been examined. The items of the scale were based on the review of this literature and the item pool of White and Horvitz (9). At the end of these studies, a total of 66 questions were prepared, 20 of which were related to demographic and cyberchondriac characteristics, and the remaining related to the different aspects of the cyberchondria concept. The statements were first evaluated by authors and then by clinical psychology experts individually in terms of structure, consistency, clarity, and repetitive expression similarities, and necessary arrangements were made by building consensus. No separate pilot study has been conducted in this context. Eventually, a 28-item final form with a 1-5 Likert type scoring (“1=Never”, “2=Seldom”, “3=Sometimes”, “4=Frequently” and “5=Always”) was developed to be prepared for validity and reliability.

Internet Addiction Scale (IAS): It is a 20-item of 1-6 Likert type scale developed by Young (33) and adapted to Turkish language by Bayraktar (34). High scores of the scale indicate higher Internet addiction. The internal consistency coefficient of Cronbach alpha of the scale was found to be 0.91 by Bayraktar (34). The same value was calculated as 0.90 in another study (35).

Brief Symptom Inventory (BSI): It is a 53-item self-assessment scale developed by Derogatis (36) for screening a variety of psychological statements through a 0-4 Likert type scoring, with a final score ranging 0 to 212. Higher total scores indicate the frequency of symptoms of the individual. Turkish adaptation of the inventory was performed with different studies in adults (37) and adolescents (38). These studies reported that the scale consisted of five factors called “anxiety”, “depression”, “negative self”, “somatization” and “hostility”. Numerous data are available on the validity and reliability of the scale (37,38).

Health Anxiety Inventory (HAI): It is an 18-item, 0-3 Likert type scale developed by Salkovskis et al. (15) to assess health anxiety. The validity and reliability study of the Turkish version of CS was conducted by Karaer et al. (39) in subjects with panic disorder; and the internal consistency coefficient of the scale was calculated as 0.91. Aydemir et al. (40) also performed a reliability and validity of the scale in individuals with somatoform disorder or panic disorders. In this study, the Cronbach alpha internal consistency coefficient was found as 0.92 (40).

Procedure

The 28-item CS has been converted into a battery by combining with other scales in different orders (to control the order effect), ensuring that the demographic form was at the beginning. Completing of the forms was made individually and participants were expected to give the most appropriate response on the scale. The participation to the study was on voluntary basis and the approval was obtained from the Ethics Committee of Ankara University. Forms were completed in about 25-30 min. Statistical analyzes were performed via SPSS-21 and AMOS-21 statistics software.

RESULTS

I. Validity

a) Explanatory and Confirmatory Factor Analysis (Structure Validation)

Structure validation of the scale was determined by explanatory factor analysis (EFA). Prior to factor analysis, the Kaiser-Meyer-Olkin (KMO) value was analyzed to test the suitability of the correlation matrix between the elements for factor analysis, which was found as 0.90. To be able to continue the factor analysis, this value must be at least 0.60 (41). In addition, Barlett Sphericity test was performed, and the data showed significant differences (χ2=3938.17, df=378, p<0.001). Thus, factor analyses were performed according to the the principal components method and varimax transformation. The initial analysis where any number of factor was not determined found five factors that had an eigenvalue above 1. As the scree-plot of the same analysis indicated the 5-factorial structure, this analysis was accepted. The factor analysis was repeated with five factorial constraints, and items with loading values greater than 0.30 were embedded into the relevant factors during this analysis. The items embedded into each factor, variance of the factors, eigenvalues, and Cronbach alpha reliability coefficients are presented in Table 1.

As shown in Table 1, the factor loadings of all items range from 0.32 to 0.81. The first factor explaining 15.56% of the variance was called as “ anxiety-increasing factors “ (6 items); the second factor explaining 12.91% as “compulsion/hypochondria”
(6 items); the third factor explaining 12.17% as “anxiety-reducing factors” (5 items); the fourth factor explaining 11.38% as “physician-patient interaction” (4 items); and the fifth factor explaining 10.31% as “non-functional Internet usage” (6 items). These five factors explained 62.34% of the total variance.

Two separate CFA studies were conducted on both the first and second sample to determine whether the five-factor model obtained in the AFA was confirmed. The path diagram, the measures of compliance, and the suggestions for correction were considered to evaluate CFA. Error variances of the items were associated in both analyzes with respect to the suggested modification indices. After each error association, the chi-square difference test was performed (42). The test results showed that these error associations made the model more compliant (p<0.05 for Δχ2). The fit indices for the models before and after the error associations are presented in Table 2.

As seen in Table 2, there were significant differences in the compliance indices between the first model and the model to which the errors were associated in both samples; and indices reached acceptable levels in recent models.

b) Measure-Related Validity

The correlation coefficients between IAS, BSI, and HAI, and the subscales formed from the CS are presented in Table 3.

As shown in Table 3, the correlation coefficients between CS total score and subscales and other scales were as expected and significant overall. These coefficients ranged from 0.53 (p<0.01) to 0.25 (p<0.01) for the CS total score, from 0.49 (p<0.01) to 0.22 (p<0.01) for the Anxiety-Increasing Factors subdimension, from 0.64 (p<0.01) to 0.30 (p<0.01) for the Compulsion/Hypochondria subdimension, from 0.25 (p<0.01) to 0.06 (p<0.05) for the Anxiety-Reducing Factors subdimension, from 0.27 (p<0.01) to 0.14 (p<0.05) for Physician-Patient Interaction subdimension, and from 0.43 (p<0.01) to 0.16 (p<0.05) for the Non-functional Internet Use subdimension.

c) Distinguishing Validity

To obtain more information about the validity of the scale, end-group analyzes were conducted comparing the cyberchondria scores of individuals with low and high scores on Internet addiction, psychological symptom, and health anxiety. Therefore, those with a 1-standard deviation (15.19) higher from the mean IAS score (41.37) were classified as “high risk Internet addiction” (72.89); those with a 1-standard deviation of low scores as “low risk Internet addiction” (38.59). Likewise, those with a 1-standard deviation (30.36) higher from the mean of the BSI score (43.62) were categorized as “high level of psychological symptom” (73.98), while those having 1-standard deviation lower were classified as “low level of psychological symptom” (13.26). Finally, those with a 1-standard deviation (7.68) higher from the mean HAI score (15.09) were grouped as “high level of health anxiety” (22.77); and those with a 1-standard deviation lower from the mean score were grouped as “low level of health anxiety” (7.41). The results of the t-test analysis are demonstrated in Table 4.

Table 4 showed that those with higher scores of IAS had significantly higher scores of total and subdomain scores of CS compared with that of lower scores of IAS. Similarly, it was demonstrated that those with higher scores of health anxiety had significantly higher scores of total and subdomain scores of CS compared with that of lower scores of health anxiety. In terms of psychological symptoms; those with higher scores of psychological symptoms had significantly higher scores of total and four of the subdomain scores of CS compared with lower scores of psychological symptoms, except the anxiety-reducing factors subdomain where no significant difference was found.

II. Reliability

Cronbach alpha reliability coefficients of CS was determined as 0.93 for the total score, 0.88 for “Anxiety-Increasing Factors” subdomain, 0.83 for “Compulsion/hypochondria” subdomain, 0.80 for “Anxiety-Reducing Factors” subdomain, and 0.84 for “Non-functional Internet Use” subdomain. The internal consistency coefficient for the subdomain of “Physician-Patient Interaction” was found as 0.78. However, the analyzes showed that the internal consistency coefficient of this subdimension would escalate to 0.80 when the item 27 (“When I brought my search to the physician, I felt uncomfortable due to being afraid of the healthcare provider’s role”) was excluded. In addition, it is noteworthy that though the item-total correlations of this item were significant when compared with other items, it also had a lower correlation coefficient (0.36). Therefore, item 27 was removed, eliciting a scale of 27 items in total.

Item-total correlations have also been examined to obtain further evidence regarding the reliability of the scale. The results are revealed in Table 1. As seen in the table, there are significant associations at the expected directions between the total score of all the subscales and consequent items. These associations varied between 0.74 (p<0.001) and 0.86 (p<0.001) for Anxiety-Increasing Factors subdomain, between 0.68 (p<0.001) and 0.77 (p<0.001) for Compulsion/Hypochondria subdomain, between 0.62 and (p<0.001) and 0.84 (p<0.001) for Anxiety-Reducing Factors subdomain, between 0.67 (p<0.001) and 0.87 (p<0.001) for Physician-Patient Interaction subdomain, and 0.70 (p<0.001) and 0.82 (p<0.001) for Non-Functional Internet Usage subdomain. Correlations of these subscales with each other ranged from 0.29 (p<0.001) to 0.65 (p<0.01).

Another method that can be used to determine the reliability of scales is the method of dividing into halves. In this respect, the correlation coefficient between the two half tests formed from the single and double items of the scale; i.e. the reliability coefficient of the dividing into halves, was determined as r=0.93 (p<0.01).

III. Demographic Variables

The gender and the age were considered as socio-demographic variables in the study and three age groups as “18-25”, “26-45” and “46-65” was created to perform statistical analysis. Thus, a multivariate analysis of variance (MANOVA) was performed in order to determine the effects of gender and age on the CS subscales, in line with the 2 (gender) x 3 (age) pattern. The analysis indicated that Wilks’ λ values were significantly different in terms of gender (Wilks’ λ=0.95, df=236, F=2.74, p<0.05, χ2=0.055); but not in terms of age (Wilks’ λ=0.95, df=472, F=1.25, p>0.05, χ2=0.026). In addition, no interaction effect was detected (Wilks’ λ=0.95, df=472, F=1.11, p>0.05, χ2=0.023).

The gender parameter showed a main effect of only on non-functional Internet use subdomain of CS (F[1-240]=5.70, p<0.05, χ2=0.023). Women had significantly higher scores (mean=14:22, SD=5.02) than that of men (mean=12.81, SD=4.36) in this subdomain.

DISCUSSION

The findings of our study suggest that CS consisting of 27 items is a valid and reliable scale that can be used in our country for an adult study population. It is noteworthy that EFA shows the items to be loaded on subscales with very high coefficients. The factor load value is a coefficient that explains the association of the items with the factors, where higher values indicate strong association of the item to the tested factor. The loading values of 0.71 and above are considered “excellent” (43), and most of the items in the current study were above this value.

First subdimension, “Anxiety-Increasing Factors”, is an increase in anxiety depending on many factors such as the layout, order, content, and reliability of the web site and the seriousness of the explanations or use of medical terminology when an individual seeks help on the Internet for a condition that is thought to be a symptom of a disease. The second subdimension, “Compulsion/hypochondria”, is defined as the prolongation of Internet searching such that it disrupts other Internet search and other daily activities originating from an anxiety that implies an ordinary symptom as an indicator of a serious disease. The third subdimension, “Anxiety-Reducing Factors”, indicates relief of the individual’s health anxiety by searching on the Internet, considering reliable sources, synthesizing relevant information, and reading web pages where individuals in similar conditions share posts. The point that should not be overlooked for this subdomain is that these behaviors provide a temporary relief, and this relaxation causes the maintenance of such behaviors, leading in the long run the individual to the beginning and development of the cyberchondria problem by keeping him/her away from a realistic solution and treatment. The fourth subdimension, “Physician-Patient Interaction”, at first sight serves to establish a good relationship between the physician and patient, hence relieving the anxiety. Nevertheless, the individual’s health-related information seeking behavior on the Internet increased such that his/her frequency of Internet utilization is increased, he or she take all the information to the doctor with a desire to discuss. As mentioned before, this subdimension can cause an interrupted physician-patient relationship. The last and fifth subdimension, the “Non-Functional Internet Usage”, refers to the use of the Internet in a way that results in a serious disease or diagnosis upon utilizing symptoms or medical terms from a health-related condition. The elevation in scores from total or each of the subdomains indicates increased level of health-related information seeking behavior and anxiety; i.e. increased level of cyberchondria.

The CFA of the scale was run on two different samples. Several values are used to test the compatibility of the data during testing of a measuring tool with CFA. The most important of these is the χ2 value. This value indicates that the developed model does not match the current data, where small or statistically non-significant values reveal good fit (43). As this value is influenced by the increase in the number of samples, the use of other fit indices is recommended (44). The parameter that should be considered in this case is the ratio of χ2 to the degrees of freedom (χ2/df). A ratio below 3 is regarded as the indicator of excellent fi, while values below 5 as acceptable fit (43,44). Other criteria used to evaluate the fitting of the data include Comparative Fit Index (CFI), Goodness of Fit Index (GFI), Root Mean Square Residual (RMR), Root Mean Square of Approximation (RMSEA), and Expected Cross Validation Index (ECVI). While acceptable values for RMSEA and RMR are ≤0.08 (44,45), these are ≥0.90 for CFI and ≥0.85 for GFI (46). The ECVI measures the discrepancy between the fitted covariance matrix in the analyzed sample and the expected covariance that will be obtained from an equally-sized sample. ECVI values is expected to be smaller than the ECVI value of the compared model (47,48). In the current study, it is notable that the ratio χ2/df is excellent for the first sample and within acceptable limits for the second sample. In addition, RMSEA, RMR, and ECVI values appear to fall within acceptable limits.

When the five factors obtained after the CFA were considered as subscales, the Cronbach alpha reliability coefficients varied between 0.80 and 0.88. In psychological studies, it is generally accepted as satisfactory value when detected as 0.60 (49) or 0.70 (50), which suggests satisfactory internal consistency coefficients of the investigated scale.

The item-total correlation coefficients obtained by calculating the correlation of the items constituting each subscale of the scale with the total score of that subscale are also found satisfactory. Several studies reported that the correlation values obtained needed to be at least 0.20 (51) or 0.30 (52), for which our findings appear relatively higher.

Another method to determine the reliability of a scale is halving method. Most appropriate method to halve a test is based on dividing the items as odd and even (53). Correlation coefficient between the two halved tests composing of odd and even-numbered items provided additional data on the reliability of the scale.

Correlation analyzes performed to determine the measure-related validity of the scale showed statistically significant associations between all subscales of the CS and other variables, except that between Anxiety-Reducing Factor subscale and BSI. Accordingly, as the level of cyberchondria elevates, the level of Internet addiction and health anxiety increases, and the severity of psychological symptoms aggravates. Available findings seem to support conceptual explanations regarding cyberchondria in the literature. Considering that repetitive and excessive utilization of the Internet for health-related help seeking is a feature of cyberchondria (2), a positive correlation between Internet addiction and the cyberchondria appears as an expected result. Moreover, consistent with our findings, several studies reported positive associations of cyberchondria with health anxiety (18,29) and scores of depression anxiety stress scale (11).

In addition, the discriminant validity of the scale was also examined, for which end-group comparisons were made. Available findings can be considered as the evidence of the validity of the scale.

Gender and age were considered as socio-demographic variables in the current study. Because of the analysis of variance (MANOVA), it was determined that women had significantly higher scores than males in the “non-functional Internet use” subscale. The studies in our country reported higher level of problematic Internet use among male high school students (54) and higher scores of Internet addiction among male university students (55) compared to their female counterparts. On the other hand, when considering the purpose of using Internet, the finding suggests that women may be using the Internet more frequently than men to conduct health-related search. This may be related to the fact that women have more health-related anxiety than do men. This is supported by the study of Barke et al. (18), reporting significantly higher cyberchondria scores of women compared with that of men.

The studies that examining the reflections of accessing medical information through Internet on patient-physician and patient-hospital relationships showed predominance of unfavorable aspects of this behavior overall, despite some positive aspects including encouraging the patient to manage own health, reducing the delay in diagnosis, drawing attention to rare diseases, etc. (56,57). For example, it has been reported that physicians became being tested by patients on medical knowledge (58), and it reduced physicians’ control on the disease and led to diminished access to accurate information about the disease (59). As search engines produce an equal number of results for similar symptoms of common or rare diseases, the information they generated constituted a biased information, which can cause anxiety and intensive concerns for the individual (9). Moreover, it was emphasized that exposure to a complicated terminology with full of medical terms led to self-diagnosis, even treatment, for what the individuals might be harmed (60). In addition to these, access to health services is becoming increasingly difficult due to the worsening economy. This is supported by the fact that self-treatment practices become widespread through the information they obtain via Internet search instead of applying to a health institution. This can undoubtedly influence the physiological and psychological health of the individuals (24,26). The major reasons of cyberchondria in these studies were thought to be the easy access to information and economic difficulties experienced, which may also well apply to our country. Considering that patient visit per physician is 4673 in Turkey according to the Turkish Statistical Institute (TUIK) data (61), it could be suggested that a reason for the individual to seek health-related information on the Internet is that they have inadequate access to an appropriate and satisfactory healthcare service.

All findings indicate the necessity of the evaluation of cyberchondria, where the measurement tools are very important. In this context, CS could be regarded as a reliable and practical scale that can be used in clinical psychology and health psychology studies owing to its low number of items and ease of scoring and interpretation. The information obtained through this scale can shed light on theoretical and practical studies to be done on health-related information-seeking on the Internet. For instance, this scale could be used during the psychological assessment or psycho-training of the clients who experience health-related anxiety associated with psychological symptoms. A detailed analysis of the clients’ responses to each item of the scale may help to gather detailed information about their characteristics related to cyberchondria (hypochondria, characteristics of information-seeking behavior, the level of influence on physician-patient relationships, etc.). Furthermore, special psycho-training programs may be developed to raise awareness that anxiety of clients may increase as their health-related Internet searching activity escalates; and the scale may be used for evaluation purposes in these programs.

The present study has several strengths. First, having a sample that includes different age groups is important for the generalizability of the findings. Second, CS has a multi-factorial structure like the other scale (31) used in our country. The other scale is predominantly about the type of symptoms (disturbed sleep cycle, decreased appetite, panic, etc.) caused by the exposure to the findings on the Internet about disease symptoms; the characteristics of the Internet search (visiting the forums or reliable sources, etc.); and issues related to healthcare professionals (increased consideration of what the doctor says, etc.). Apart from these, CS provides detailed information about that the increased anxiety of the Internet findings regarding the disease symptoms is related to what features of the browsed pages (such as mentioning of serious explanations, use of complex medical terms, contents of pages, sorting of pages, etc.). CS could also give data about what kind of behaviors (such as reading online forums, synthesizing, searching reliable sources, etc.) help to decrease anxiety. Moreover, it provides information on both compulsion and hypochondria. Furthermore, it focuses on the potential effects of the cyberchondria on physician-patient interaction. For example, it gives information about whether the patient’s search on the Internet would facilitate his or her communication with the physician, or how he or she would feel him/herself while sharing the findings based on the physician’s role. Briefly, while the other scale (31) enables us to obtain more information about the symptoms of cyberchondria and the negative effects of these symptoms, CS aims to measure the level of cyberchondria by focusing on the features of the web pages which increase or decrease the symptoms and the characteristics of information-seeking behavior. It also provides information regarding hypochondria. In addition to all these, the reliability coefficients of the CS subscales were found to be higher than those of the other scale (31). These features could well distinguish this scale from other scales reported in the literature.

Undoubtedly, there are also some limitations of the study. For example, the sample was drawn from inhabitants of Ankara province. It is recommended to collect data from different cities in future studies. It should also be remembered that the findings reported in this study were obtained from a non-clinical sample and the scale has not yet been studied on a clinical sample. The vast majority of the sample is made up of university graduates and therefore the effect of the education was not tested. Literature showed that cyberchondria tended to occur more frequently among younger populations with higher level of education and socioeconomical status compared to the general population (62), though it is suggested that the effect of these variables be examined in future studies.

In conclusion, considering all these limitations, it should be noted that CS is a self-assessment measurement tool, where the collected data needs to be used along with the data from observations and the interview.

Informed Consent: Written consent was obtained from the patients.

Peer-review: Externally peer-reviewed..

Conflict of Interest: Authors declared no conflict of interest.

Financial Disclosure: This study was supported by the Ankara University Scientific Research Projects (Project No: 16B0649001).

REFERENCES

1.Greene JA, Kesselheim AS. Pharmaceutical marketing and the new social media. N Engl J Med 2010; 363:2087-2089. [CrossRef]

2.Starcevic V, Berle D. Cyberchondria: towards a better understanding of excessive health-related Internet use. Expert Rev Neurother 2013; 13:205-213. [CrossRef]

3.Amara R, Bodenhorn K, Cain M, Carlson R, Chambers J, Cypress D, Dempsey H, Falcon R, Garces R, Garrett J, Gasper D, Sanstad KH, Holt M, Kirsch S, Kuehn N, Kuiper H, Kyrouz E, Mittman R, Morrison E, Morrison I, Nilsen G, Pascali M, Robertson A, Runde D, Sarasohn-Kahn J, Schmid G, Wilson C, Yu K. Health and Health Care 2010: The Forecast, The Challenge. Second ed., Institute for the Future, San Francisco, CA: Jossey-Bass, 2003.

4.Fox S. The Social Life of Health Information. Washington, DC: Pew Internet and American Life Project, 2011.

5.Higgins O, Sixsmith J, Barry MM, Domegan C. A literature review on health information-seeking behaviour on the web: a health consumer and health professional perspective. Stockholm: ECDC, 2011.

6.Huberty J, Dinkel D, Beets MW, Coleman J. Describing the use of the internet for health, physical activity, and nutrition information in pregnant women. Matern Child Health J 2013; 17:1363-1372. [CrossRef]

7.Lemire M, Sicotte C, Paré G. Internet use and the logics of personal empowerment in health. Health Policy 2008; 88:130-140. [CrossRef]

8.Aiken M, Kirwan G, Berry M, O’Boyle CA. The age of cyberchondria. Royal College of Surgeons in Ireland Student Medical Journal 2012; 5:71-74.

9.White RW, Horvitz E. Cyberchondria: studies of the escalation of medical concerns in web search. ACM Trans Manag Inf Syst 2009; 27:23. [CrossRef]

10.Benigeri M, Pluye P. Shortcomings of health information on the Internet. Health Promot Int 2003; 18:381-386. [CrossRef]

11.McElroy E, Shevlin M. The development and initial validation of the Cyberchondria Severity Scale (CSS). J Anxiety Disord 2014; 28:259-265. [CrossRef]

12.Norr AM, Capron DW, Schmidt NB. Medical information seeking: impact on risk for anxiety psychopathology. J Behav Ther Exp Psychiatry 2014; 45:402-407. [CrossRef]

13.Cline RJW, Haynes KM. Consumer health information seeking on the Internet: the state of the art. Health Education Research 2001; 16:671-692. [CrossRef]

14.Abramowitz JS, Olatunji BO, Deacon BJ. Health anxiety, hypochondriasis, and the anxiety disorders. Behav Ther 2007; 38:86-94. [CrossRef]

15.Salkovskis PM, Rimes KA, Warwick HM, Clark DM. The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med 2002; 32:843-853. [CrossRef]

16.Fergus TA, Russell LH. Does cyberchondria overlap with health anxiety and obsessive–compulsive symptoms? An examination of latent structure and scale interrelations. J Anxiety Disord 2016; 38:88-94. [CrossRef]

17.Ivanova E. Internet addiction and cyberchondria-their relationship with well-being. Journal of Education Culture and Society 2013; 1:57-70.

18.Barke A, Bleichhardt G, Rief W, Doering BK. The Cyberchondria Severity Scale (CSS): German validation and development of a short form. Int J Behav Med 2016; 23:595-605. [CrossRef]

19.Norr AM, Albanese BJ, Oglesby ME, Allan NP, Schmidt NB. Anxiety sensitivity and intolerance of uncertainty as potential risk factors for cyberchondria. J Affect Disord 2015; 174:64-69. [CrossRef]

20.Norr AM, Allan NP, Boffa JW, Raines AM, Schmidt NB. Validation of the Cyberchondria Severity Scale (CSS): replication and extension with bifactor modeling. J Anxiety Disord 2015; 31:58-64. [CrossRef]

21.Norr AM, Oglesby ME, Raines AM, Macatee RJ, Allan NP, Schmidt NB. Relationships between cyberchondria and obsessive-compulsive symptom dimensions. Psychiatry Res 2015; 230:441-446. [CrossRef]

22.Muse K, McManus F, Leung C, Meghreblian B, Williams JM. Cyberchondriasis: fact or fiction? A preliminary examination of the relationship between health anxiety and searching for health information on the internet. J Anxiety Disord 2012; 26:189-196. [CrossRef]

23.Baumgartner SE, Hartmann T. The role of health anxiety in online health information search. Cyberpsychol Behav Soc Netw 2011; 14:613-618. [CrossRef]

24.White RW, Horvitz E. Experiences with web search on medical concerns and self-diagnosis. AMIA Annu Symp Proc 2009; 2009:696-700.

25.Doherty-Torstrick ER, Walton KE, Fallon BA. Cyberchondria: parsing health anxiety from online behavior. Psychosomatics 2016; 57:390-400. [CrossRef]

26.White RW, Horvitz E. Web to world: predicting transitions from self-diagnosis to the pursuit of local medical assistance in web search. AMIA Annu Symp Proc 2010; 2010:882-886.

27.Ravdin LD. Guide for clinicians in the age of cyberchondria. J Int Neuropsychol Soc 2008; 14:912-916.

28.Keller GL, Padala PR, Petty F. Clinical pearls to manage cyberchondriacs. Prim Care Companion J Clin Psychiatry 2008; 10:75-76. [CrossRef]

29.Fergus, TA. The Cyberchondria Severity Scale (CSS): an examination of structure and relations with health anxiety in a community sample. J Anxiety Disord 2014; 28:504-510. [CrossRef]

30.Uzun SU. The level of cyberchondria and associated factors among Pamukkale University employees. Thesis of specialty in medicine, Pamukkale University, Denizli, Turkey, 2016. (Turkish)

31.Uzun SU, Akbay B, Ozdemir C, Zencir M. The validity and reliability of the cyberchondria severity scale and its short form in university students. Abstrack Book of the National Public Health Congress, 2017, p241. (Turkish)

32.Erkus A. Scientific Research Process. Fourth Ed. Ankara: Seckin Yayincilik, 2013, 122-123. (Turkish)

33.Young KS. Internet addiction: the emergence of a new clinical disorder. Cyberpsychol Behav 1998; 1:237-244. [CrossRef]

34.Bayraktar F. The role of internet use during adolescence. Master of science thesis, Ege University, Izmir, Turkey, 2001. (Turkish)

35.Batigun AD, Hasta D. Internet addiction: an evaluation in terms of loneliness and interpersonal relationship styles. Anatolian Journal of Psychiatry 2010; 11:213-219. (Turkish)

36.Derogatis LR. The Brief Symtom Inventory (BSI): Administration, scoring and procedures manual. Second ed., USA: Clinical Psychometric Research Inc., 1992.

37.Sahin NH, Durak A. Brief Symptom Inventory (BSI): adapting to the Turkish youth. Turkish Journal of Psychology 1994; 9:44-56. (Turkish)

38.Sahin NH, Batigun AD, Ugurtas S. The validity, reliability and factor structure of the Brief Symptom Inventory (BSI). Turk Psikiyatri Derg 2002; 13:125-135. (Turkish)

39.Karaer-Karapicak O, Aktas K, Aslan S. Turkish validity and reliability study of Health Anxiety Inventory (Weekly Short Form) in cases of panic disorders. Turkish Journal of Clinical Psychiatry 2012; 15:41-48. (Turkish)

40.Aydemir O, Kirpinar I, Sati T, Uykur B, Cengisiz C. Reliability and validity of the Turkish version of the Health Anxiety Inventory. Noro Psikiyatr Ars 2013; 50:325-331. (Turkish) [CrossRef]

41.Worthington RL, Whittaker TA. Scale development research: a content analysis and recommendations for best practices. Couns Psychol 2006; 34:806-838. [CrossRef]

42.Tabachnick BG, Fidell LS. Using multivariate statistics. Fourth ed., Boston: Allyn and Bacon, 2001.

43.Cokluk O, Sekercioglu G, Buyukozturk S. Multivariance Statistical SPSS and LISREL Applications for Social Sciences. Ankara: Pegem, 2012. (Turkish)

44.Sumer N. Structural equality models: basic concepts and model practices. Türk Psikoloji Yazıları 2000; 3:49-74. (Turkish)

45.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999; 6:1-55. [CrossRef]

46.Cole DA. Utility of confirmatory factor analysis in test validation research. J Consult Clin Psychol 1987; 55:584-594. [CrossRef]

47.Loehlin JC. Path Models in Factor, Path, and Structural Equation Analysis. Latent Variable Models. Fourth ed., ABD: Lawrence Erlbaum Associates, 2004, 202-254.

48.Byrne BM. Testing for the Factorial Validity of a Theoretical Construct. Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming. Second ed., New York: Routledge, 2010, 74-82.

49.Aron A, Aron EN. Making Sense of Advanced Statistical Procedures in Research Articles. Statistics For Psychology. Third ed., New Jersey: Prentice Hall, 2003, 599-637.

50.Erkus A. Scientific Research Process for Behavioral Sciences: Paper-Based Data Collection Tools. Fourth Ed. Ankara: Seckin Yayinevi, 2013, 149-168. (Turkish)

51.Field A. Discovering Statistics Using SPSS. Third ed., London: Sage, 2009.

52.Nunnally JC, Bernstein IH. Psychometric Theory. Third ed., New York: McGraw-Hill, 1994.

53.Erkus, A. Letters on psychometry: historical origins of measurement and psychometry, its reliability, validity, item analysis, attitudes: its components and measurement. Ankara: Turk Psikologlar Dernegi Yayinlari, 2003.

54.Zorbaz O, Tuzgol-Dost M. Examination of problematic internet use of high school student in terms of gender, social anxiety and peer relations. Hacettepe University Journal of Education 2014; 29:298-310. (Turkish)

55.Batigun AD, Kilic N. Associations between internet addiction and personal characteristics, social support, psychological symptoms, and some socio-demographic features. Turkish Journal of Psychology 2011; 26:1-10. (Turkish)

56.Bouwman MG, Teunissen QG, Wijburg FA, Linthorst GE. ‘Doctor Google’ending the diagnostic odyssey in lysosomal storage disorders: parents using internet search engines as an efficient diagnostic strategy in rare diseases. Arch Dis Child 2010; 95:642-644. [CrossRef]

57.Kim S, Southwell B. Driven to the internet? Satisfaction with physician-patient communication as a predictor of consumers’ online information seeking. Paper presented at the annual meeting of the International Communication Association, Suntec City, Singapore, 2010. http://citation.allacademic.com/meta/p403793_index.html

58.Lewis T. Seeking health information on the internet: lifestyle choice or bad attack of cyberchondria? Media Cult Soc 2006; 28:521-539. [CrossRef]

59.Lowrey W, Anderson WB. The impact of internet use on the public perception of physicians: a perspective from the sociology of professions literature. Health Commun 2006; 19:125-131. [CrossRef]

60.Benigeri M, Pluye P. Shortcomings of health information on the Internet. Health Promot Int 2003; 18:381-386. [CrossRef]

61.Turkish Statistics Institute. The number of physicians, the number of individuals per physician, and the number of visits per physician during 2009-2015. http://www.tuik.gov.tr/PreIstatistikTablo.do?istab_id=1612 Accessed December 11, 2017. (Turkish)

62.Fox S. Online Health Search. Pew Internet and American Life 2006. http://www.pewinternet.org/files/old-media/Files/Reports/2006/PIP_Online_Health_2006.pdf.pdf Accessed December 11, 2017.