ABSTRACT

Objective: To translate the Perme Intensive Care Unit Mobility Score and the ICU Mobility Scale (IMS) into Portuguese, creating versions that are cross-culturally adapted for use in Brazil, and to determine the interobserver agreement and reliability for both versions. Methods: The processes of translation and cross-cultural validation consisted in the following: preparation, translation, reconciliation, synthesis, back-translation, review, approval, and pre-test. The Portuguese-language versions of both instruments were then used by two researchers to evaluate critically ill ICU patients. Weighted kappa statistics and Bland-Altman plots were used in order to verify interobserver agreement for the two instruments. In each of the domains of the instruments, interobserver reliability was evaluated with Cronbach's alpha coefficient. The correlation between the instruments was assessed by Spearman's correlation test. Results: The study sample comprised 103 patients-56 (54%) of whom were male-with a mean age of 52 ± 18 years. The main reason for ICU admission (in 44%) was respiratory failure. Both instruments showed excellent interobserver agreement (κ > 0.90) and reliability (α > 0.90) in all domains. Interobserver bias was low for the IMS and the Perme Score (−0.048 ± 0.350 and −0.06 ± 0.73, respectively). The 95% CIs for the same instruments ranged from −0.73 to 0.64 and −1.50 to 1.36, respectively. There was also a strong positive correlation between the two instruments (r = 0.941; p < 0.001). Conclusions: In their versions adapted for use in Brazil, both instruments showed high interobserver agreement and reliability.

Keywords: Physical therapy modalities; Intensive care units; Translations; Validation studies.

RESUMO

Objetivo: Realizar a tradução e a validação cultural para a língua portuguesa falada no Brasil e determinar a concordância e a confiabilidade dos instrumentos Perme Intensive Care Unit Mobility Score (designado Perme Escore) e ICU Mobility Scale (designada Escala de Mobilidade em UTI, EMU). Métodos: Os processos de tradução e adaptação cultural seguiram as seguintes etapas: preparação, tradução, reconciliação, síntese, tradução reversa, revisão, aprovação e pré-teste. Após esses processos, as versões em português dos dois instrumentos foram utilizadas por dois pesquisadores na avaliação de pacientes críticos em UTI. O índice kappa ponderado e a disposição gráfica de Bland-Altman foram utilizados para verificar a concordância entre os instrumentos. O coeficiente alfa de Cronbach foi utilizado para verificar a confiabilidade entre as respostas dos avaliadores dentro de cada domínio dos instrumentos. A correlação entre os instrumentos foi verificada pelo teste de correlação de Spearman. Resultados: A amostra foi composta por 103 pacientes, sendo a maioria homens (n = 56; 54%), com média de idade = 52 ± 18 anos. O principal motivo de internação nas UTIs foi insuficiência respiratória (em 44%). Os dois instrumentos apresentaram excelente concordância interobservador (κ > 0,90) e confiabilidade (α > 0,90) em todos os domínios. Constatou-se um baixo viés interobservador na EMU e no Perme Escore (−0,048 ± 0,350 e −0,06 ± 0,73, respectivamente). Os IC95% para os mesmos instrumentos variaram, respectivamente, de −0,73 a 0,64 e de −1,50 a 1,36, respectivamente. Além disso, verificou-se alta correlação positiva entre os dois instrumentos (r = 0,941; p < 0,001). Conclusões: As versões dos dois instrumentos apresentaram alta concordância e confiabilidade interobservador.

Palavras-chave: Objetivo: Realizar a tradução e a validação cultural para a língua portuguesa falada no Brasil e determinar a concordância e a confiabilidade dos instrumentos Perme Intensive Care Unit Mobility Score (designado Perme Escore) e ICU Mobility Scale (designad

INTRODUCTION

Early mobilization is part of the rehabilitation process for ICU patients and is currently considered a way to prevent ICU-acquired muscle weakness and worsening of physical function.(1,2) Some studies have associated the practice of early mobilization with decreased duration of mechanical ventilation and reduced length of ICU and hospital stays, as well as with the promotion of functional improvement in ICU survivors.(1,3,4)

There are currently 26 published instruments that purport to assess function in ICU patients. Of those, the Functional Independence Measure and the Barthel index have been used both in clinical practice and in research. (1,5) However, few of these instruments were developed and validated to assess function and/or mobility in ICU patients. In fact, only 6 were developed specifically for the ICU setting and have published clinimetric data. These instruments are the Physical Function in Intensive care Test scored, the Chelsea Critical Care Physical Assessment tool, the Perme Inten-sive Care Unit Mobility Score, the Surgical intensive care unit Optimal Mobilization Score, the ICU mobility scale, and the Functional Status Score for the ICU.(6) However, none of them are considered "gold standard" to assist multidisci-plinary teams in quantifying the patient's degree of mobility in a rapid, easy, and objective way.(7,8) In addition, there are conditions extrinsic to the patient that affect the patient's mobility in bed, such as the presence of access ports, lines, and chest tubes, which can be interpreted as a barrier to mobility, and this presence is not scored or considered in most instruments.(5,9-11)

Taking into account such limitations, Perme et al.(12) developed a specific instrument for measuring improvement in mobility status, with a view to standardizing the evaluation of ICU patients-the Perme Intensive Care Unit Mobility Score-hereafter referred to as the Perme Score, which is an instrument that objectively measures the mobility status of ICU patients, starting with the ability to follow commands and culminating in the distance walked in two minutes. This mobility instrument is scored from 0 to 32 and comprises 15 items grouped into 7 categories: mental status; potential mobility barriers; functional strength; bed mobility; transfers; gait (with or without assistive devices); and endurance. In this instrument, a high score indicates high mobility and a decreased need for assistance. In contrast, a low score indicates low mobility and an increased need for assistance.(12)

With similar objectives to those of the Perme Score, Hodgson et al.(8) also developed an instrument for measuring mobility status in ICU patients objectively. Designated the ICU Mobility Scale (IMS), this single-domain instrument is scored from 0 to 10, with a score of 0 meaning low mobility (interpreted as a patient being capable of performing only passive exercises in bed) and a score of 10 meaning high mobility (interpreted as a patient being capable of inde-pendent ambulation, without aid).(8)

The clinical utility of a tool has to be determined on the basis of a logical assessment of its validation, reliability, and applicability.(13) In order to choose the best tool that can effectively assess the functional changes that will occur in the patient during the ICU stay, health care professionals and researchers should consider which tools have clini-metric data that are more robust and appropriate for the functional outcomes that they want to analyze.(6,14) To date, neither the Perme Score nor the IMS has been appropriately translated and validated for use in Brazil, taking into account the language and cultural differences. Therefore, the objective of the present study was to translate these two ICU mobility instruments into Portuguese, creating versions that are cross-culturally adapted for use in Brazil, and to determine inter-rater agreement and reliability for both versions.

METHODS

The present study was approved by the Research Ethics Committee and the Comissão de Análises de Projetos de Pesquisa (CAPPesq, Committee for the Analysis of Research Projects) of the University of São Paulo School of Medi-cine Hospital das Clínicas (Ruling no. 657.496). The translated instruments were evaluated at two clinical ICUs (10 beds) and one surgical ICU (20 beds) of the University of São Paulo School of Medicine Hospital das Clínicas Central Institute, in the city of São Paulo, Brazil, between April and June of 2015. In the phase of instrument testing, since physical therapy evaluation was part of the routine care at those ICUs, being performed several times a day, the health care professionals who participated in the study, scoring and comparing the instruments, gave written in-formed consent, rather than the patients.

The methodology for translating and cross-cultural adapting and validating the instruments followed a rigorous pro-cess, in accordance with current guidelines for the translation and cross-cultural adaptation of instruments.(15,16) The following steps were performed: 1) Preparation: the author of the project contacted the authors of the original instru-ments and obtained the rights to use, translate, and cross-culturally validate the instruments; 2) Translation from English into Portuguese: the instruments were independently translated into the target language by two translators who were native speakers of Portuguese and fluent in English, one of whom was familiar with the instruments and was aware of the objective of the present study and the other of whom was not familiar with the instruments; 3) Rec-onciliation and synthesis: the two initial Portuguese-language versions were compared, item by item, with the origi-nal English-language versions by two physical therapists who were familiar with the instruments. Any existing dis-crepancies were analyzed and discussed by three researchers, leading to the production of a second Portuguese-language version for each of the two instruments; 4) Back-translation: the second Portuguese-language version of each of the two instruments was sent to two translators who were native speakers of English and fluent in Portuguese, neither of whom had contact with the original English-language versions, for back-translation(15-17); 5) Review and harmonization of the back-translation: the back-translated versions of the instruments were compared with their original English-language versions by a review committee comprising three researchers, in order to identify potential discrepancies and make the necessary adjustments, item by item, thereby producing the final back-translated version of each of the two instruments; 6) Approval from the authors of the original instruments: the final back-translated versions were sent to the authors of the original instruments for evaluation and comments on their consistency. An expert committee comprising three physical therapists analyzed the evaluations and comments of the authors of the original instruments, incorporating their suggestions, and thereby produced the final Portuguese-language version of each of the two instruments; and 7) Pre-test: raters were trained on the administration and scoring of the final Portu-guese-language version of each of the two instruments. After training, a pilot study involving 40 patients was con-ducted in which two raters administered the two instruments following the methodology described in the original articles; during these evaluations, the raters could discuss the scores and the difficulties in administering each in-strument.(13)

Data were collected by two raters, one of whom was a senior (> 5 years of experience) physical therapist (rater 1) and one of whom was a junior (< 5 years of experience) physical therapist (rater 2). The two raters performed the scoring according to the rules of the two mobility instruments, which were administered after an initial evaluation made by the ICU physical therapist. While one of the raters evaluated the patient, the other one only observed the procedure, without having any physical contact with the patient. Each rater was responsible for 50% of the evalua-tions, and the functions of rater and observer were swapped every two patients. Both raters completed the scoring sheet of the Perme Score and of the IMS according to the highest activity level. In an attempt to avoid biases, the scoring sheets were completely separate and there was no communication between the raters.(7,8)

Data on age, gender, reason for ICU admission, mechanical ventilation use, vasoactive drug use, and score on the Simplified Acute Physiology Score 3 were collected to determine the clinical characteristics of the population.
The sample size was calculated with a level of significance of 5% and a power of 80%, taking into account that the instruments could be equal (50%) or not (50%). This is possible through the use of the Bernoulli probability distribu-tion; in addition, we considered a delta of 10%, that is, the probability of equality could range from 40% to 60%, and found that a sample size of 100 individuals was required. Statistical analysis was performed with the Statistical Pack-age for the Social Sciences, version 17 (SPSS Inc., Chicago, IL, USA).

The clinical characteristics of the patients were descriptively expressed as mean and standard deviation, median and interquartile range, or proportion, depending on data type and normality of distribution. The level of inter-rater agreement in the scoring of each instrument was determined using weighted kappa statistics and 95% CI. Inter-rater reliability (internal consistency) in scoring was determined using Cronbach's alpha coefficient. For the Perme Score, inter-rater agreement and reliability were assessed individually for each domain (items 1 to 15). In addition, Bland-Altman plots were used to determine inter-rater agreement in total score (sum of all domains) both for the Perme Score and the IMS. The proportions of evaluations with minimum scores (floor effect) and maximum scores (ceiling effect) were also calculated. Finally, the Kolmogorov-Smirnov test and Levene's test were used to determine normali-ty and homoscedasticity, respectively. Since these principles were not met, Spearman's correlation coefficient was used to test the correlation between the two instruments.(7,8,13) For this correlation analysis, we used the values from the two raters.

RESULTS

Table 1 shows the clinical characteristics of the patients evaluated in the present study. Slightly more than half (54%; n = 56) of our sample was male and 67% (n = 69) of the patients were admitted for clinical reasons, the most prevalent being respiratory disorders (n = 45). Mechanical ventilation was present in 36% (n = 37) of the cases, and vasoactive drug use occurred in 51% (n = 53). Appendices show the Portuguese-language versions of the IMS and the Perme Score, both of which were translated from the original instruments. They are available online at www.jornaldepneumologia/link
 



Table 2 shows the inter-rater agreement (kappa statistics and 95% CI) and reliability (internal consistency, Cronbach's alpha coefficient) for the IMS and for each domain of the Perme Score. The IMS showed excellent inter-rater agreement and reliability. In addition, the inter-rater agreement for each item of the Perme Score ranged from 78% to 100%, and the inter-rater reliability (Cronbach's alpha coefficient) ranged from 88% to 100%, meaning that there was excellent inter-rater agreement and reliability for all items. Figure 1 presents the Bland-Altman plots for the IMS and for the total score on the Perme Score. Inter-rater bias was low both for the IMS (−0.048 ± 0.35) and the Perme Score (−0.06 ± 0.73). The 95% CIs ranged from −0.73 to 0.64 for the IMS and from −1.50 to 1.36 for the Perme Score.
 

 




The floor effect for the IMS and the Perme Score was found to be 36% and 20%, respectively. The ceiling effect for the IMS and the Perme Score was found to be 6% and 3%, respectively.

The mean completion time for the scoring sheets was two minutes for the Perme Score and less than one minute for the IMS. There was also a strong positive correlation between the use of the two instruments in the evaluation of the patients (r = 0.941; p < 0.001).

DISCUSSION

For the present study, two instruments for evaluating mobility in ICU patients were carefully translated into Portu-guese and validated for use in Brazil, with technical and semantic equivalence having been achieved between the original versions and the Portuguese-language versions. Our results show that, in their versions adapted for use in Brazil, both instruments showed high inter-rater agreement and reliability after a brief period of familiarization and training with them. In addition, there was a strong positive correlation between the two instruments.

Performing physical therapy in critically ill patients is currently in the spotlight, with numerous publications com-menting on its prevalence and benefits.(18-20) In this context, some instruments for evaluating function and mobility have been developed specifically for this population in order to improve physical therapy care in terms of performing and progressing the exercises in ICU patients according to the mobility milestones that each individual can reach.(8) To date, as mentioned above, no instrument for evaluating mobility in ICU patients has been cross-culturally adapted for use in Brazil. The careful translation and cross-cultural validation of such an instrument makes it possible for health care professionals nationwide to have access to a tool that can improve the quality of care to critically ill pa-tients in the ICU, as well as allowing the comparison of results across studies conducted in different countries.(15)

In their versions adapted for use in Brazil, both the IMS and the Perme Score showed excellent inter-rater agree-ment and reliability (κ > 0.9 and α > 0.9 for most domains). Although the group who developed the Perme Score reported moderate to high reliability, our study reported an even higher level of inter-rater reliability.(7,12) One possi-ble explanation for this finding it that, in our study, the sample size was larger than those of the two previous stud-ies.(7,12) In view of the ease of learning and ease of use of the Perme Score, any disagreements in score had a lesser impact in our study than in the validation studies for the original English-language version of the instrument.(7,12) Our study also reported higher inter-rater reliability for the Portuguese-language version of the IMS than that reported in the validation study for the original English-language version of the instrument.(8) In this case, the collection method may have affected the result. In our study, the raters performed the scoring simultaneously but in an independent fashion, each being blinded to the scoring by the other rater, whereas in the study by Hodgson et al.,(8) each assessor separately evaluated the patient at 30-min intervals.(8)

Interestingly, the IMS showed excellent Inter-rater reliability and agreement, although its single domain is scored from 0 to 10, an 11-point range. Although it is a greater range than that in each domain of the Perme Score, the IMS comprises mobility milestones that are clear and can easily be evaluated by the rater.

Item 6 in the Perme Score, "potential mobility barriers-continuous intravenous infusion", was the one showing the lowest inter-rater reliability and agreement in our study. Although some patients had venous access for administra-tion of saline or drugs, in some cases, the infusion was not being administered at the time of evaluation, which may have confused the raters in scoring this item. However, we emphasize that, even with this likely difference, the re-ported inter-rater reliability and agreement were higher than 75-80%.

The Bland-Altman plots showed high inter-rater score agreement and low inter-rater score variability for the IMS and for the sum of all domains of the Perme Score. Since the evaluation of each domain had shown excellent inter-rater agreement and reliability, the sum of all domains did not change this behavior. On the basis of the 95% CIs, the maximum inter-rater difference was 2 points for the Perme Score and less than 1 point for the IMS.

It was expected that the scores on the two instruments would be highly correlated, given that both instruments measure the same property and therefore should show a similar behavior.

Finally, instrument floor and ceiling effects of 15% or less are considered acceptable. In our study, floor effects were found to be higher for the two instruments (20% and 36% for the Perme Escore and the IMS, respectively). Knowing that these instruments purport to assess functioning, floor effects were expected to be higher than normal, given the high incidence of sedated or unconscious patients in ICUs. Although data collection was performed at three different ICUs in order to try to minimize this drawback, in 35% of the evaluations, the patients were unconscious or had lethargic responses, which made it impossible to perform more functional tasks or mobilizations at the time of evaluation. The lower floor effect of the Perme Score as compared with that of the IMS can be explained by the scor-ing of the different domains of the former, such as patient cooperation, presence of pain, and presence of barriers to mobilization. Although they are not mobilization aspects per se, they end up affecting the ease or difficulty of mobili-zation.

Some limitations of our study should be taken into account. First, no clinimetric analyses were performed other than inter-rater reliability and agreement testing and inter-instrument correlation analysis. However, the primary objective of the present study was the cross-cultural validation of the instruments for use in Brazil. We recognize that the ap-plicability of the instruments and their predictive and concurrent validity have yet to be tested. Although the clinimet-ric properties of the IMS were tested against those of the Physical Function in Intensive care Test scored,(6) the same was not true for the Perme Score. Second, in our study, the two participating raters who were tested for inter-rater reliability and agreement were physical therapists. Knowing that such health care professionals are directly related to the process of functional evaluation and early mobilization of critically ill patients, analysis of inter-rater reliability and agreement involving such professionals was fundamental. However, we cannot state that the characteristics reported in the present study can be obtained by the other health care professionals who comprise multidisciplinary ICU teams, such as nurses and physicians, and will eventually use the instruments. Finally, the evaluations took place concurrently. Therefore, aspects such as tone of voice, personal approach, and instruction of patients, all of which may differ from one health care professional to another, were not fully evaluated in our study. However, each rater was responsible for half of the evaluations, which to some extent minimized this effect.

Therefore, we conclude that the Brazilian Portuguese-language versions of the IMS and the Perme Score were ap-propriately translated and cross-culturally validated, following strict guidelines, and can be used in Brazil. Both ver-sions showed excellent inter-rater agreement and reliability.

REFERENCES

1. Schweickert WD, Pohlman MC, Pohlman AS, Nigos C, Pawlik AJ, Esbrook CL, et al. Early physical and occupational therapy in mechanically ventilated, critically ill patients : a randomised controlled trial. Lancet. 2009;373(9678):1874-82. http://dx.doi.org/10.1016/S0140-6736(09)60658-9
2. Burtin C, Clerckx B, Robbeets C, Ferdinande P, Langer D, Troosters T, et al. Early exercise in critically ill patients enhances short-term functional recovery. Crit Care Med. 2009;37(9):2499-505. http://dx.doi.org/10.1097/CCM.0b013e3181a38937
3. Morris PE, Goad A, Thompson C, Taylor K, Harry B, Passmore L, et al. Early intensive care unit mobility therapy in the treatment of acute respiratory failure. Crit Care Med. 2008;36(8):2238-43. http://dx.doi.org/10.1097/CCM.0b013e318180b90e
4. Needham DM, Korupolu R, Zanni JM, Pradhan P, Colantuoni E, Palmer JB, et al. Early physical medicine and rehabilitation for patients with acute respiratory failure : a quality improvement project. Arch Phys Med Rehabil. 2010;91(4):536-42. http://dx.doi.org/10.1016/j.apmr.2010.01.002
5. Montagnani G, Vagheggini G, Panait Vlad E, Berrighi D, Pantani L, Ambrosino N. Use of the Functional Independence Measure in people for whom weaning from mechanical ventilation is difficult. Phys Ther. 2011;91(7):1109-15. http://dx.doi.org/10.2522/ptj.20100369
6. Parry SM, Denehy L, Beach LJ, Berney S, Williamson HC, Granger CL. Functional outcomes in ICU - what should we be using ? - an observational study. Crit Care. 2015;19:127. http://dx.doi.org/10.1186/s13054-015-0829-5
7. Nawa RK, Lettvin C, Winkelman C, Evora PR, Perme C. Initial interrater reliability for a novel measure of patient mobility in a cardiovascular intensive care unit. J Crit Care. 2014;29(3):475.e1-5. http://dx.doi.org/10.1016/j.jcrc.2014.01.019
8. Hodgson C, Needham D, Haines K, Bailey M, Ward A, Harrold M, et al. Feasibility and inter-rater reliability of the ICU Mobility Scale. Hear Lung. 2014;43(1):19-24. http://dx.doi.org/10.1016/j.hrtlng.2013.11.003
9. MAHONEY FI, BARTHEL DW. FUNCTIONAL EVALUATION: THE BARTHEL INDEX. Md State Med J. 1965;14:61-5.
10. Skinner EH, Berney S, Warrillow S, Denehy L. Development of a physical function outcome measure (PFIT) and a pilot exercise training protocol for use in intensive care. Crit Care Resusc. 2009;11(2):110-5.
11. Thrush A, Rozek M, Dekerlegand JL. The clinical utility of the functional status score for the intensive care unit (FSS-ICU) at a long-term acute care hospital: a prospective cohort study. Phys Ther. 2012;92(12):1536-45. http://dx.doi.org/10.2522/ptj.20110412
12. Perme C, Nawa RK, Winkelman C, Masud F. A tool to assess mobility status in critically ill patients: The Perme Intensive Care Unit Mobility Score. Methodist Debakey Cardiovasc J. 2014;10(1):41-9. http://dx.doi.org/10.14797/mdcj-10-1-41
13. Nassar Junior AP, Pires Neto RC, de Figueiredo WB, Park M. Validity, reliability and applicability of Portuguese versions of sedation-agitation scales among critically ill patients. Sao Paulo Med J. 2008;126(4):215-9. http://dx.doi.org/10.1590/S1516-31802008000400003
14. Parry SM, Granger CL, Berney S, Jones J, Beach L, El-Ansary D, et al. Assessment of impairment and activity limitations in the critically ill : a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med. 2015;41(5)744-62. http://dx.doi.org/10.1007/s00134-015-3672-x
15. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-91. http://dx.doi.org/10.1097/00007632-200012150-00014
16. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures : report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104. http://dx.doi.org/10.1111/j.1524-4733.2005.04054.x
17. Tavares MG, Pizzichini MM, Steidle LJ, Nazário NO, Rocha CC, Perraro MC, et al. The Asthma Control Scoring System: translation and cross-cultural adaptation for use in Brazil. J Bras Pneumol. 2010;36(6):683-92.
18. Morris PE. Moving our critically ill patients: mobility barriers and benefits. Crit Care Clin. 2007;23(1):1-20. http://dx.doi.org/10.1016/j.ccc.2006.11.003
19. Lima NP, Silva GM, Park M, Pires-Neto RC. Mobility therapy and central or peripheral catheter-related adverse events in an ICU in Brazil. J Bras Pneumol. 2015;41(3):225-30. http://dx.doi.org/10.1590/S1806-37132015000004338
20. Denehy L, Skinner EH, Edbrooke L, Haines K, Warrillow S, Hawthorne G, et al. Exercise rehabilitation for patients with critical illness: a randomized controlled trial with 12 months of follow-up. Crit Care. 2013;17(4):R156. http://dx.doi.org/10.1186/cc12835
21. Terwee CB, Bot SD, de Boer MR, Van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42. http://dx.doi.org/10.1016/j.jclinepi.2006.03.012