Instruments for Measuring the Influence of Visual Persuasion: Validity and Reliability Tests

Authors

  • Nurulhuda Ibrahim School of Multimedia Technology and Communication, Universiti Utara Malaysia
  • Mohd Fairuz Shiratuddin School of Engineering and Information Technology, Murdoch University, Australia
  • Kok Wai Wong School of Engineering and Information Technology, Murdoch University, Australia

DOI:

https://doi.org/10.26417/ejser.v4i1.p25-37

Keywords:

reliability test, validity test; visual persuasion measures, PLS-SEM

Abstract

In User Experience (UX) research, the instruments are often measured by means of rating scales such as Likert scale and semantic differential scale. The validity of the findings and conclusions rely heavily on the instruments used in the questionnaires. This paper provides the assessment of the validity and reliability of a new set of measures to evaluate the influence of visual persuasion on web users. The instruments will be used to assess web users' perceptions of credibility, engagement, informativeness, satisfaction, social influences, usability, and visual aesthetic. Firstly, 85 items are pilot tested by expert and novice users in an offline and online settings. Secondly, the exploratory factor analysis is carried out in which 44 items representing 12 latent variables are reduced to 39 items with some of the latent variables are combined into one. The results show: Kaiser-Meyer-Olkin (KMO) of 0.901, significant Bartlett’s test, communalities range between 0.470 - 0.829, nine factors (also known as the latent variables) emerged with eigenvalues greater than 1, explaining more than 60 percent of the total variance, factor loadings of 0.466 and above, factors correlations of less than 0.7, and Cronbach’s alphas are well above the limit of 0.70. Finally, a confirmatory factor analysis is carried out on the first-order and second-order latent variables using the PLS-SEM. The instruments exceed the minimum requirement of the assessments for the convergent validity, discriminant validity, reliability and collinearity. The findings suggest that the proposed 39 items are valid and reliable for measuring the persuasiveness of visual persuasion.

References

Albert, W., & Tullis, T. (2013). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics (2nd Editio.). Elsevier Science.

Chu, H.-L., Deng, Y.-S., & Chuang, M.-C. (2014). Persuasive Web Design in e-Commerce. In F.-H. Nah (Ed.), HCI in Business SE - 47 (Vol. 8527, pp. 482–491). Springer International Publishing. doi:10.1007/978-3-319-07293-7_47

Cialdini, R. B. (2007). Influence: The Psychology of Persuasion. Collins (Revised ed., Vol. 55). HarperBusiness;

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1–9. doi:10.1.1.110.9154

Cugelman, B., Thelwall, M., & Dawes, P. L. (2009). The Dimensions of Web Site Credibility and Their Relation to Active Trust and Behavioural Impact. Communications of the Association for Information Systems, 24, 455–472.

Cyr, D. (2013). Website design, trust and culture: An eight country investigation. Electronic Commerce Research and Applications, 12, 373–385. doi:10.1016/j.elerap.2013.03.007

Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research (JMR). Feb1981, 18(1), 39–50. 12p. 1 Diagram. doi:10.2307/3151312

Gaskin, J. (2012a). Data screening. Gaskination’s StatWiki. Retrieved March 30, 2015, from http://statwiki.kolobkreations.com

Gaskin, J. (2012b). Exploratory Factor Analysis. Gaskination’s StatWiki. Retrieved April 5, 2015, from http://statwiki.kolobkreations.com

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Multivariate Data Analysis. Prentice Hall.

Hao, J.-X., Tang, R., Yu, Y., Li, N., & Law, R. (2015). Visual Appeal of Hotel Websites: An Exploratory Eye Tracking Study on Chinese Generation Y. In I. Tussyadiah & A. Inversini (Eds.), Information and Communication Technologies in Tourism 2015 SE - 44 (pp. 607–620). Springer International Publishing. doi:10.1007/978-3-319-14343-9_44

Hayduk, L. A., & Littvay, L. (2012). Should researchers use single indicators, best indicators, or multiple indicators in structural equation models? BMC Medical Research Methodology. doi:10.1186/1471-2288-12-159

Hooper, D. (2012). Exploratory factor analysis. In H. (Ed. . Chen (Ed.), Approaches to Quantitative Research - Theory and its Practical Application: A Guide to Dissertation Students. Cork, Ireland: Oak Tree Press.

Horvath, J. (2011). Persuasive Design: It’s Not Just about Selling Stuff . In A. Marcus (Ed.), Design, User Experience, and Usability. Theory, Methods, Tools and Practice (Vol. 6770, pp. 567–574). Springer Berlin / Heidelberg. doi:10.1007/978-3-642-21708-1_63

Kim, H. (2008). Persuasive Architecture of Tourism Destination Websites: An Analysis of First Impression.

Kim, H., & Fesenmaier, D. R. (2008). Persuasive Design of Destination Web Sites: An Analysis of First Impression. Journal of Travel Research, 47(1), 3–13. doi:10.1177/0047287507312405

Kock, N. (2010). Using WarpPLS in E-collaboration Studies: An Overview of Five Main Analysis Steps. Int. J. E-Collab., 6(4), 1–11. doi:10.4018/jec.2010100101

Kock, N. (2011). Using WarpPLS in e-collaboration studies: Mediating effects, control and second order variables, and algorithm choices. International Journal of E-Collaboration, 7(3), 1–13. doi:10.4018/jec.2011070101

Kock, N. (2014). A note on how to conduct a factor-based PLS-SEM analysis.

Kock, N. (2015a). One-Tailed or Two-Tailed P Values in PLS-SEM? Int. J. E-Collab., 11(2), 1–7. doi:10.4018/ijec.2015040101

Kock, N. (2015b). WarpPLS 5.0 User Manual. Laredo, TX: ScriptWarp Systems.

Leimeister, S. (2010). IT outsourcing governance: Client types and their management strategies. IT Outsourcing Governance: Client Types and Their Management Strategies. doi:10.1007/978-3-8349-6303-1

Mackenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35(2), 293–334. Retrieved from http://dl.acm.org/citation.cfm?id=2017510npapers3://publication/uuid/E4D3717C-7F3F-4791-8835-141D4309976B

Schmiedel, T., vom Brocke, J., & Recker, J. (2014). Development and validation of an instrument to measure organizational cultures’ support of business process management. Information & Management, 51(1), 43–56. doi:10.1016/j.im.2013.08.005

Sekaran, U., & Bougie, R. (2010). Research methods for business: a skill-building approach. Chichester: Wiley.

Small, R. V, & Arnone, M. P. (1998). Website Motivational Analysis Checklist for Business (WebMAC Business). New York: The Motivation Mining Company.

Small, R. V, & Arnone, M. P. (2000). Website Motivational Analysis Checklist (WebMAC): Professional. New York: Motivation Mining.

Tang, L. (2009). Destination websites as advertising: An application of Elaboration Likelihood Model. ProQuest, UMI Dissertations Publishing.

Tenenhaus, M., Vinzi, V. E., Chatelin, Y. M., & Lauro, C. (2005). PLS path modeling. Computational Statistics and Data Analysis, 48(1), 159–205. doi:10.1016/j.csda.2004.03.005

Winn, W., & Beck, K. (2002). The Persuasive Power of Design Elements on an E-Commerce Web Site. Technical Communication, 49(1), 17–35.

?

Downloads

Published

2022-03-03

How to Cite

Ibrahim, N., Shiratuddin, M. F., & Wong, K. W. (2022). Instruments for Measuring the Influence of Visual Persuasion: Validity and Reliability Tests. European Journal of Social Science Education and Research, 9(1), 44–68. https://doi.org/10.26417/ejser.v4i1.p25-37