Dual coding theory

Overview

Dual coding theory was proposed by Paivio(1991) that verbal and non-verbal representations constitute the two main channels of human cognition. It is underlined that both verbal and non-verbal information processing are crucial for the storing, processing, and extraction of information. Dual coding theory of multimedia learning is a prominent and crucial theory that supports screen-based computer media learning, meaning that learning through the simultaneous presentation of multiple formats such as picture animations and audio narration as learning materials increases the learning effect of the learner. It helps learners make referential connections between visual and verbal information and transfer knowledge to different contexts (Mayer,1994).

Evidence

Based on the theory proposed by Paivio, Mayer (1994) conducted two experiments to extend the dual coding theory of multimedia learning in terms of problem-solving ability and individual differences. The study results support the dual coding theory of multimedia learning's contiguity impact, and the contiguity effect was more substantial for students with greater spatial ability. This is due to the fact that individuals with good spatial abilities may more readily construct a visual representation base on animation when working memory resources are constrained. The students who participated in experiments had no corresponding prior experience, and students who received instructional material provided with both animated films and narratives outperformed those who received continuous presentations or no instruction on the problem-solving test. At the same time, it is clear that inexperienced students in a given domain also have strong contiguity effects because dual-channel learning compensates for the new knowledge difficulty of retrieval from long-term memory and the establishment of connections, while students with domain-specific expertise are able to complete retrieval in long-term memory on their own. In my opinion,  this definitely demonstrates the value of using DCT-based instructional media materials in assisting with knowledge transfer, and that there is a noticeable facilitation effect for transfer test questions that involve problem-solving. Especially for learners with great spatial abilities or no prior experience, the use of two-channel media resources enables a high level of interaction between the visual and verbal representations, aiding in learners’ understanding and retention of new information.

Design Implications

Instructional media materials utilizing DCT are also applicable to beginning second-language learners. Wong & Samudra (2019) use within-subjects design to examine the effects of verbal and visual media on 43 preschool bilingual learners (DLLs) from varied ethnic origins learning a non-native language. DLLs would watch "Sesame Street" vocabulary instructional videos. Dual coding ensures visual and auditory consistency, such as when an athlete appears on the screen and a character says, "That's the athlete, Elmo!". DLLs were able to detect more words taught on-screen measured by eye-tracing when the material was dually encoded compared to non-dual coding (visual–auditory incongruence), particularly if they're English-challenged.

Another example is the starter learning mode on Duolingo, which presents vocabulary, images, and audio all at once in accordance with the two channels indicated by DCT, like the word "apple" is accompanied by an image of an apple and an audio pronunciation. For novices learning L2 on-screen, consistent visual and aural information sources might act as crucial compensatory scaffolding to enhance learners' recognition and understanding of vocabulary.

Challenges

Screen-based learning with engaging graphics and sounds may help learners focus and learn, but there are still some significant problems to consider. However, working memory processing is restricted, and the integration of auditory and visual information will be limited by the cognitive load(Sweller et al., 1998). If dual-channel instructional media content contains a significant number of stunning images as well as audio and video files, whether the auditory or visual channel provide too much information to be processed, resulting in cognitive overload and limiting learners' learning? Therefore, how can instructional designers make better use of DCT without putting themselves in danger of having to deal with extraneous cognitive load? I believe that we should consider the above questions before adopting dual-channel instructional media materials.

References

Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149–210. https://doi.org/10.1007/bf01320076 Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389–401. https://doi.org/10.1037/0022-0663.86.3.389 Sweller, J., van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive Architecture and Instructional Design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/a:1022193728205 Wong, K. M., & Samudra, P. G. (2019). L2 vocabulary learning from educational media: Extending dual-coding theory to dual-language learners. Computer Assisted Language Learning, 34(8), 1182–1204. https://doi.org/10.1080/09588221.2019.1666150