ABSTRACT

Objective: To translate, cross-culturally adapt to Brazilian Portuguese, and evaluate the reliability of the Telemedicine Satisfaction Questionnaire (TSQ). Methods: This cross-sectional study involved patients from the Smoking Prevention and Cessation Center (PrevFumo) who participated in at least four of the eight scheduled remote meetings with the PrevFumo psychologist in 2020, 2021, or 2022. Participants were contacted by telephone and asked to answer the 14 questions of the TSQ three times at intervals of 7 or 10 days. Results: We assessed 53 patients (73.3% women). The mean age was 49.7 ± 10.2 years. The mean smoking history was 35.32 ± 24.8 pack-years. Of the 53 patients evaluated, 30.2% had completed high school or had some higher education, and 32.1% were classified as socioeconomic class B2 (A being the highest and E being the lowest). Forty-nine (92.5%) of the patients attended all eight meetings. The TSQ with only three answer options showed high reliability, with approximately 90% agreement after three applications. Patients were satisfied with telemedicine. Conclusions: The TSQ is rapidly applied, is easy to complete, and showed high reliability in our patient sample. Patients declared that they were satisfied with their telemedicine experience.

Keywords: Telemedicine; Patient satisfaction; Surveys and questionnaires; Cross-cultural comparison; Smoking cessation.

INTRODUCTION
 
The Pan American Health Organization defines telemedicine as the remote, real-time delivery of health care services by any health professional using information and communication technologies to diagnose, treat, and prevent diseases.(1)
 
In Brazil, the use of telemedicine began in 1994, first being employed by cardiologists for remote electrocardiogram examinations.(2) On December 27, 2022, law no. 14,510 amended law no. 8,080 (from September 19, 1990), to authorize and regulate the practice of telemedicine throughout the country. Telemedicine, known as telessaúde in Brazil, comprises the remote delivery of services related to all health professions regulated by competent agencies of the federal executive branch.
 
Several questionnaires assess patient satisfaction with telemedicine. The following are the most widely used: the Telehealth Usability Questionnaire, which assesses the feasibility of implementing telemedicine services(3); the Telemedicine Satisfaction Questionnaire (TSQ)(4); the Florida Patient Acceptance Survey(5); the Telemedicine Satisfaction and Usefulness Questionnaire(6); the Patient Satisfaction Questionnaire Short-Form(7); the Patient Satisfaction with Physician(8); the Service User Technology Acceptability Questionnaire,(9) which assesses the beliefs of uses regarding the acceptability of telemedicine; and the Telehealth Satisfaction Scale, developed for patients with memory disorders.(10)
 
To our knowledge, there have been few studies evaluating patient satisfaction with telemedicine in Brazil, and none have used a translated, cross-culturally adapted questionnaire according to guidelines.(11,12) Although Dias et al.(13) used a questionnaire to evaluate the satisfaction of patients receiving telemedicine for the treatment of headaches, the description encompassed only the domains and possibilities of answers (“yes” and “no”), with no detailing of the questions. Severini et al.(14) assessed the satisfaction of adult patients who used telemedicine, although their study lacked descriptions regarding how the questionnaire was adapted. Brandão et al.(15) reported a satisfaction questionnaire to assess the acceptance and impact of telemedicine at a referral center for special immunobiologicals; however, the questionnaire questions were not described. Macharet et al.(16) assessed the feasibility of telemedicine in urogynecology using only seven questions from a questionnaire originally consisting of 14. Dias et al.(17) investigated the efficiency of real-time telerehabilitation in patients with Parkinson’s disease and found good patient satisfaction with telemedicine. However, their study sample comprised only 20 participants.
 
At the Paulista School of Medicine Smoking Prevention and Cessation Center (aka, PrevFumo), group orientation began remotely immediately after COVID-19 was declared a pandemic. However, when the pandemic was declared over, we had to decide whether the group orientation should return to in-person or continue being conducted remotely. To date, the in-person consultations have returned only for the initial visit. In this context, because of a lack of information regarding the satisfaction of our patients with remote group orientation, we felt compelled to assess their level of satisfaction before deciding whether to recommence the in-person group meetings. A literature review about satisfaction with telemedicine resulted in the choice of the TSQ because of its easy application and objective questions to assess the level of satisfaction with telemedicine and the reasons for the satisfaction. (4) Nevertheless, before it could be used in Brazil, the TSQ would have to be translated and adapted to Brazilian culture.
 
The primary objective of this study was to translate, cross-culturally adapt, and validate the TSQ for use in Brazil. Secondary objectives were to analyze the relationships that satisfaction with telemedicine has with nicotine addiction, smoking history, education level, socioeconomic class, symptoms of anxiety, depressive symptoms, and quality of life.
 
METHODS
 
Patients were invited by telephone to participate in this cross-sectional study. The study was approved by the Research Ethics Committee of the Federal University of São Paulo/Hospital São Paulo (Reference no: 64593722.0.0000.5505), in the city of São Paulo, Brazil. All participating patients gave written informed consent, either by e-mail or by text message. We included adult patients treated via the PrevFumo, regardless of sex and level of education, who participated in at least four of the eight scheduled remote group orientations.
 
Initially, the questionnaire was translated by one of the investigators who was fluent in English and Brazilian Portuguese.(11,12) In the cross-cultural adaptation phase, the initial Brazilian Portuguese-language version was presented to 10 patients from our outpatient clinics, who discussed the words that best expressed what the questionnaire proposed. This version was further discussed by a multidisciplinary team, including a physiotherapist, a psychologist, a nurse, and a physician. The cross-culturally adapted version was then back-translated to English by another person and compared to the original English version to evaluate the similarity.(11,12)
 
After a given patient had been enrolled, the following data were collected from medical records of the initial consultation at PrevFumo: sociodemographic characteristics including age, gender, education, and socioeconomic class according to the Brazilian economic classification criteria(18); smoking history; degree of nicotine addiction, determined with the Fagerström test(19); respiratory symptoms; motivation to stop smoking; degree of anxiety, depression, or both, determined with the Hospital Anxiety and Depression Scale (HADS)(20); and quality of life, determined with the World Health Organization Quality of Life Instrument, brief version (WHOQOL-BREF).(21) The Brazilian economic classification criteria classify individual socioeconomic status into six classes from A (the highest) to E (the lowest). The Fagerström test comprises six questions with scores ranging from 0 (no addiction) to 10 (high addiction). The HADS has 14 questions that evaluate the probability of a diagnosis of anxiety or depression, with scores ranging from 0 to 21. The WHOQOL-BREF has 26 questions with a maximum total score of 130, higher values being associated with better quality of life.
 
The TSQ consists of 14 questions about satisfaction with telemedicine,(4) answered on a five-point Likert scale (Table 1): 1 = strongly disagree; 2 = disagree; 3 = indifferent; 4 = agree; and 5 = strongly agree. To assess the reliability of the questionnaire, we recruited a sample of 50 participants to answer the 14 TSQ questions by phone call at three different time points, one-week apart.(22) The same investigator conducted all three interviews, which ensured test-retest reliability.

 
Statistical analyses were conducted using the R software, version 4.2.2.(23) Data normality was verified with the Shapiro-Wilk test. Values are presented as absolute and relative frequencies (for categorical variables) or as mean and standard deviation (for continuous variables). Intraclass correlation coefficients (ICCs) were calculated in order to compare the reliability of TSQ responses among the three applications, and results were classified as poor (< 0.5), moderate (0.5-0.75), good (> 0.75-0.9), or excellent (> 0.9).(24) For correlation analysis between the TSQ score and the nonparametric variables age, education, socioeconomic class, Fagerström test score, smoking history, anxiety, depression, attendance to the virtual education meetings, and quality of life, Spearman’s test was used. For comparison of the nonparametric variable sex, the Mann-Whitney test was used. There is no consensus on the interpretation of correlation values, but it is widely accepted that values under 0.4 should be considered indicative of a weak correlation. Values of p < 0.05 were considered statistically significant.
 
RESULTS
 
The final sample consisted of 53 participants, 73.3% of whom were female (Table 2). The mean age was 49.7 ± 10.2 years. Of the 53 patients evaluated, 16 (30.2%) had completed high school or had some higher education. The most common socioeconomic class (in 32.1%) was B2 (29-37 points). A total of 92.5% of participants connected to all eight meetings. The mean total score of the WHOQOL-BREF was 12.3 ± 2.49 (51.9% ± 15.5%), and the mean total score of the Fagerström test was 5.81 ± 2.28 (39.6% were highly addicted). The mean smoking history was 35.3 ± 24.8 pack-years. On the HADS, the mean anxiety score was 10.8 ± 4.4 (45.3% had probable anxiety), whereas the mean depression score was 8.2 ± 3.7 (49.1% had probable depression).

 
Reliability analysis
 
Table 3 presents the ICC between the three applications of the TSQ. Reliability was poor for 8 of the 14 questions. In addition, reliability values were better in the second and third interviews than in the first interview, probably because of the poor-to-moderate concordance found.

 
Descriptive analysis for the grouped score with five answer options
 
Table 4 shows the frequency of combinations found in the three interviews, from which we considered a total of 742 answers (53 participants times 14 questions). The percentage of concordance for the answers “strongly agree” (number 5) and “agree” (number 4) in the three interviews was 46% and 16.7%, respectively. The following five most frequent combinations included only the answers “strongly agree” and “agree” (numbers 4 and 5).

 
Descriptive analysis for the grouped score with three answer options
 
During the interviews, participants repeatedly reported not seeing differences between the answers “strongly disagree” and “disagree” and between “strongly agree” and “agree”. In this sense, we reanalyzed data considering “strongly disagree” and “disagree” as one answer and “strongly agree” and “agree” as another answer; thus, resulting in three options: “disagree” (number 1), “indifferent” (number 2), and “agree” (number 3). With this new classification, we observed that 89.9% of participants answered “agree” (number 3) in all three interviews (Table 5).
 
Correlations between TSQ score and sociodemographic variables
 
No association was found between the TSQ score and sex on the Mann-Whitney test (W = 320.5, p = 0.332). On Spearman’s correlation test, the TSQ score was not found to correlate with age (r = −0.18, p = 0.196), education (r = −0.17, p = 0.21), socioeconomic class (r = 0.21, p = 0.140), attendance to telemedicine educational meetings (r = 0.24, p = 0.080), nicotine addiction (r = 0.17, p = 0.212), smoking history (r = 0.07, p = 0.602), anxiety (r = 0.07, p = 0.602), depression (ρ = 0.07, p = 0.602), and quality of life (ρ = −0.05, p = 0.714).
 
DISCUSSION
 
For this study, we translated the TSQ to Brazilian Portuguese, cross-culturally adapted it for use in Brazil, and evaluated its reliability. We found that all of the participants were satisfied with telemedicine and that the TSQ was reliable, rapidly applied, easily completed, and easy to understand.
 
The translation and cross-cultural adaptation of a questionnaire should follow the guidelines established by Beaton et al.(11) and Wild et al.(12) Depending on the complexity of a questionnaire, more than one translator can perform the translation and back-translation. (12) When translating a questionnaire into the local language, it is recommended that the senior author of the original questionnaire be consulted to discuss and resolve questions regarding the interpretation of terms or phrases. In the cross-cultural adaptation phase, the version translated into the local language must be presented and discussed with persons with the same status the questionnaire was created, in order to determine which words best express what the questionnaire will assess. For this phase, a multidisciplinary team joined the investigators to discuss with patients which terms would be the best for the Brazilian Portuguese-language version of the TSQ. The translated version should then be back-translated by a different person, compared with the original version to evaluate similarity,(11,12) and presented to the senior author of the original version for acceptance or suggestions.(11,12) The translation and cross-cultural adaption of the TSQ for use in Brazil encompassed all of those phases. However, we were unable to contact the first or the senior authors of the original TSQ, despite sending messages via internet and attempting to contact the university. Nevertheless, we do not believe that the lack of contact limited the translation, given that the statements in the questionnaire were quite clear.
 
Few questionnaires assessing satisfaction with telemedicine can be found in the Brazilian literature. Studies similar to ours have used different questionnaires without clear descriptions regarding their development or translation. Although Dias et al.(17) assessed satisfaction with telemedicine by questioning sociodemographic parameters, transportation, and travel time to the hospital, no description was presented regarding the number of patients satisfied with the care given by the health care professional. Those authors also did not describe the translation and cross-cultural adaptation. Although the TSQ has been previously used in Brazil for assessing the perspectives of legal guardians regarding consultations with patients under 18 years of age, only 7 of the 14 questions were asked and no information was given concerning the translation.(16) Likewise, Brandão et al.(15) applied a satisfaction survey for the use of immunobiologicals, although the questionnaire used was not specified.
 
The present study assessed reliability by applying the Brazilian Portuguese-language version of the TSQ at three different time points. Reliability is the repeatability or reproducibility of a measure or variable, and it is commonly assessed by applying a questionnaire at least twice.(25-27) However, reliability increases with sample size and the number of questionnaire applications. For a sample size of 50 participants with three repetitions, the confidence intervals are narrower than, for instance, a sample of 15 participants with four repetitions. Therefore, the number of questionnaire applications is important in a study design to reduce the typical error.(22).
 
Participants completed the TSQ by choosing one of the five answers on the Likert scale for each question. However, most of them stated that the answers “strongly disagree” and “disagree” and the options “strongly agree” and “agree” were similar and difficult to understand as different options. Considering that the answers “strongly agree” and “agree” expressed the same feeling (i.e., acceptance of what was considered in the question), we merged the two answers into one and found that most (89.9%) of the participants were satisfied with their telemedicine experience. Although Likert scales typically present five answer options, a different number of options can be considered. For example, in one study that cross-culturally adapted an instrument to investigate the perception of health professionals about teleconsultation, seven options were given.(28) Likewise, five options were presented to patients undergoing treatment for type 2 diabetes,(4) whereas three options were given to evaluate the responsiveness of a system by applying a usability script to three devices(29): notebooks, tablets, and smartphones. It is possible that the level of education influenced the number of answer options; that is, participants with a high level of education might be more capable of sorting out a more precise answer than are those with no formal education. Nevertheless, we observed no differences among different education levels in terms of the answers chosen.
 
For questions 12, 13, and 14 of the TSQ, which refer directly to satisfaction with telemedicine, the positive response rate was high (87.7%, 94.3%, and 94.3%, respectively) after three interviews. The reasons why patients were satisfied with telemedicine were assessed through questions 7, 9, 10, and 11, which also presented high reliability (90.1%, 94.3%, 84.9%, and 90.6%, respectively). The ICC analysis was hampered by the large percentage of answers grouped into two responses, considering the five Likert scale options.
 
The novel finding of this study was the high rate of satisfaction with telemedicine by our patients. Telemedicine has been used for over 30 years,(30) and the COVID-19 pandemic recently led to in its adoption worldwide. With the closure of the outpatient clinic, virtual consultations became an alternative for communication between health care professionals and patients. Our smoking cessation clinic maintained the group orientation through telemedicine, but information on patient satisfaction or whether they would prefer to go back to the previous system (in-person group meetings) was lacking. Despite the high satisfaction rate, our results may apply only to certain consultations. Knowledge about the physical health of patients was optional in the smoking cessation group, because the meetings focused on advice and reinforcement techniques.
 
One limitation of our study was the initial refusal of participants to respond to calls and questions at three different time points to answer the same questions; they considered the repetition unnecessary. This was mitigated by explaining the reasons for the repetition and emphasizing the need for appropriate guidance to increase the reliability of the study. Another limitation was the application of the TSQ to patients who required only guidance and motivation to stop smoking. Therefore, other specialties should use the Portuguese-language version of the TSQ that has been cross-culturally adapted for use in Brazil to determine whether patients were satisfied with telemedicine. A third possible limitation was the lack of a test-retest reliability analysis.
 
Despite its development in 2002 and use in various studies, the TSQ is not widely used in Brazil, probably because previous translations were not performed in accordance with the standards.(11) In addition, the use of telemedicine became widespread in Brazil in 2020 (during the COVID-19 pandemic) and was regulated only in 2022.(31) With the increasing development of telemedicine in Brazil, the TSQ is a reliable tool to assess the level of satisfaction of patients regarding health care services.
 
One of the strengths of the present study was the large sample size and application of the TSQ at three different time points. This design increased the reliability of results. Another strength was the high reliability regarding satisfaction with telemedicine. In addition, no significant difference was observed between the TSQ score and the variables assessed, indicating that it may be applied to any individual, regardless of age, gender, education level, socioeconomic class, symptoms of depression or anxiety, and quality of life.
 
The Portuguese-language version of the TSQ, cross-culturally adapted for use in Brazil, appears to be easily and quickly applied, and patients considered the use of telemedicine acceptable in the smoking cessation group meetings. This acceptability might have been influenced by the fact that telemedicine is reliable, saves time, provides adequate attention, and meets health care needs. Applying the TSQ at three different time points, with three answer options, showed high test-retest reliability.
 
AUTHOR CONTRIBUTIONS
 
MEL: data collection, writing, editing, and descriptive analysis; SSN: writing and editing; ACF: data collection and essay; JRJ: writing, descriptive analysis, and translation.
 
CONFLICTS OF INTEREST
 
None declared.
 
REFERENCES
 
1.           Polizuk AK, Gómez AJS. Aplicaciones de telecomunicaciones en salud en la subregión andina: Telemedicina. Washington DC: Organización Panamericana de La Salud, OPS/OMS; 2003. Adobe Acrobat document, 50p.]. Available from: http://git.unicauca.edu.co/ehas/docs/Salvador2005/LibroORAS/Resumen-Telemedicina-Aplicaciones%20de%20telecomunicaciones%20en%20salud%20en%20la%20subregion%20andina.pdf
2.           Consórcio de Inovação na Gestão Pública (CIGA) [HOMEPAGE on the Internet] Florianópolis: CIGA; [cited 2023 Apr 2]. A origem da telemedicina no Brasil. Available from: https://consorciociga.gov.br/a-origem-da-telemedicina-no-brasil/
3.           Parmanto B, Lewis AN Jr, Graham KM, Bertolet MH. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil. 2016;8(1):3-10. https://doi.org/10.5195/ijt.2016.6196
4.           Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study. J Telemed Telecare. 2003;9(1):46-50. https://doi.org/10.1258/135763303321159693
5.           Burns JL, Serber ER, Keim S, Sears SF. Measuring patient acceptance of implantable cardiac device therapy: initial psychometric investigation of the Florida Patient Acceptance Survey. J Cardiovasc Electrophysiol. 2005;16(4):384-390. https://doi.org/10.1046/j.1540-8167.2005.40134.x
6.           Bakken S, Grullon-Figueroa L, Izquierdo R, Lee NJ, Morin P, Palmas W, et al. Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc. 2006;13(6):660-667. https://doi.org/10.1197/jamia.M2146
7.           Marshall GN, Hays RD. The Patient Satisfaction Questionnaire Short-Form (PSQ-18). Santa Monica, CA: Rand; 1994.
8.           Agha Z, Schapira RM, Laud PW, McNutt G, Roter DL. Patient satisfaction with physician-patient communication during telemedicine. Telemed J E Health. 2009;15(9):830-839. https://doi.org/10.1089/tmj.2009.0030
9.           Torbjørnsen A, Småstuen MC, Jenum AK, Årsand E, Ribu L. The Service User Technology Acceptability Questionnaire: Psychometric Evaluation of the Norwegian Version. JMIR Hum Factors. 2018;5(4):e10255. https://doi.org/10.2196/10255
10.        Morgan DG, Kosteniuk J, Stewart N, O’Connell ME, Karunanayake C, Beever R. The telehealth satisfaction scale: reliability, validity, and satisfaction with telehealth in a rural memory clinic population. Telemed J E Health. 2014;20(11):997-1003.. https://doi.org/10.1089/tmj.2014.0002
11.        Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-3191. https://doi.org/10.1097/00007632-200012150-00014
12.        Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. https://doi.org/10.1111/j.1524-4733.2005.04054.x
13.        Dias L, Martins B, Pinto MJ, Rocha AL, Pinto M, Costa A. Headache teleconsultation in the era of COVID-19: Patients’ evaluation and future directions. Eur J Neurol. 2021;28(11):3798-3804. https://doi.org/10.1111/ene.14915
14.        Severini RDSG, Oliveira PC, Couto TB, Simon Junior H, Andrade APM, Nanbu DY, et al. Fast, cheap and feasible: Implementation of pediatric telemedicine in a public hospital during the Covid-19 pandemic. J Pediatr (Rio J). 2022;98(2):183-189. https://doi.org/10.1016/j.jped.2021.05.007
15.        Brandão LGP, da Costa MD, Martins PS, de Jesus-Junior SCA, de Aguiar DF, de Lemos AS, et al. Telemedicine in the National Immunization Program (Brazil): A promising tool. Vaccine X. 2022;11:100188. https://doi.org/10.1016/j.jvacx.2022.100188
16.        Macharet DV. Implementação de Telemedicina em Uroginecologia: Estudo de Viabilidade [dissertation]. Belo Horizonte: Universidade Federal de Minas Gerais; 2022.
17.        Dias AE, Limongi JC, Barbosa ER, Hsing WT. Voice telerehabilitation in Parkinson’s disease. Codas. 2016;28(2):176-181. https://doi.org/10.1590/2317-1782/20162015161
18.        Associação Brasileira de Empresas de Pesquisa (ABEP) [homepage on the Internet]. São Paulo: ABEP. Critério de Classificação Econômica Brasil. Available from: http://www.abep.org/
19.        Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: a revision of the Fagerström Tolerance Questionnaire. Br J Addict. 1991;86(9):1119-1127. https://doi.org/10.1111/j.1360-0443.1991.tb01879.x
20.        Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361-370. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x
21.        Skevington SM, Sartorius N, Amir M. Developing methods for assessing quality of life in different cultural settings. The history of the WHOQOL instruments. Soc Psychiatry Psychiatr Epidemiol. 2004;39(1):1-8. https://doi.org/10.1007/s00127-004-0700-5
22.        Hopkins WG. Measures of reliability in sports medicine and science. Sports Med. 2000;30(1):1-15. https://doi.org/10.2165/00007256-200030010-00001
23.        R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021. Available from: https://www.R project.org/
24.        Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research [published correction appears in J Chiropr Med. 2017 Dec;16(4):346. doi: 10.1016/j.jcm.2017.10.001]. J Chiropr Med. 2016;15(2):155-163. https://doi.org/10.1016/j.jcm.2017.10.001
25.        Souza TC, Jardim JR, Jones P. Validação do questionário do hospital Saint George na doença respiratória (SGRQ) em pacientes portadores de doença pulmonar obstrutiva crônica no Brasil. J Pneumol. 2000;26(3):119 128 https://doi.org/10.1590/S0102-35862000000300004 https://doi.org/10.1590/S0102-35862000000300004
26.        Camelier A, Rosa F, Jones P, Jardim JR. Validation of the Airways questionnaire 20 - AQ20 in patients with chronic obstructive pulmonary disease (COPD) in Brazil . J Pneumol. 2003;29(1):28-35 https://doi.org/10.1590/S0102-35862003000100007
27.        Rê A, Fonseca FR, Queiroz AP, Reis CMD, Bahl MM, Kocks J, et al. Brazilian version of the Clinical COPD Questionnaire, administered by interview: reliability and validity measurement properties. J Bras Pneumol. 2021;47(3):e20200371. https://doi.org/10.36416/1806-3756/e20200371
28.        Pedroso A. Adaptação transcultural de questionário sobre teleconsulta na área da saúde: versão em português do Health Optimum Telemedicine Acceptance Questionnaire [dissertation]. Curitiba: Universidade Tuiuti do Paraná; 2023.
29.        Ensina LA, Lee HD, Takaki WSR, Maciejewski NAR, Spolaôr N, Wu FC. Heuristics-based responsiveness evaluation of a telemedicine computational web system. IEEE Latin America Transactions. 2019;17(3):444-452. Retrieved from: https://latamt.ieeer9.org/index.php/transactions/article/view/139 https://doi.org/10.1109/TLA.2019.8863315
30.        Preston J, Brown FW, Hartley B. Using telemedicine to improve health care in distant areas. Hosp Community Psychiatry. 1992;43(1):25-32. https://doi.org/10.1176/ps.43.1.25
31.        Senadonotícias [homepage on the Internet]. Brasil: Senado Federal; [updated 2022 Dec 28; cited 2024 Jan 1]. Lei autoriza telessaúde com autonomia para profissionais e consentimento de pacientes. Available from: https://www12.senado.leg.br/noticias/materias/2022/12/28/lei-autoriza-telessaude-com-autonomia-para-profissionais-e-consentimento-de-pacientes:~:text=O%20presidente%20Jair%20Bolsonaro%20sancionou,quarta%2Dfeira%20