Concept testing for Learning/Teaching

[Jinen]

Post teaching a topic, I consider it is very useful for the professor/trainer to conduct a small concept related test to understand how well have the teachings translated into knowledge of the trainees/students.

In a case where the next topic of teaching is dependent on the previous topic,the results of the test helps all the more to fine tune your next lecture's deliverables.

I would like to know your ideas on the strategy of concept testing. Also, if there are better and known alternatives then it would be nice to discuss.

[Joseph McFadden]

Jinen,

Great Discussion Topic. Not one that I believe is in the dialog often enough. Concept testing, is for the reason you've stated extremely important, both for the student and the flow of the instruction. Everyone probably has horror stories of being in a class with someone who just didn't "get it" earlier in the course. They then needed to absorb vast quantities of the instructors time, and by extension all the participants, to simply keep up.

That said, I find the most important aspect of concept testing strategy is to, in Steven Covey's words, "Begin with the end in mind." Establishing a clear set of leaning objectives to govern both course development and testing development is vital.

Objectives establish the standard against which the student grasp of the material is measured. Properly developed test instruments perform the measurement. Also, it's quite likely that developing the objectives and tests early also helps guide the course development to prevent development and delivery "scope drift."

Thanks,

Joe McFadden

[Jinen]

Joe,

"Begin with the end in mind" - well said. I agree that properly developed test instruments will perform the measurements.

Well, do you think analysis engine to understand the measurement of the class for a particular topic or subject as a whole is critical? Do you think such an indicator can assist the trainer to enhance his teaching methodology and deliver better results? Or do you think it will assist an instructional designer to understand gaps in the course design if any?

Thanks,
Jinen

[Joseph McFadden]

Absolutely essential for all the reasons you've mentioned. If I understand "analysis engine" in this context, as either tools / techniques to help understand the testing results across a broader time period or the larger set of participants, then yes it's critical. Anything to help understand the statistics and perhaps more importantly to gain statistical control of the "analysis / development / delivery / testing / analysis /..." cycle will just make the training and potential retention better.

I teach software-use in an intense "hands-on" environment. And while there are no formal test instruments (pen & paper, Q&A), I use a combination of performance indicators generated from the participants work in the exercises. I look over the results each evening and use the information to guide review / delivery the following day. Also this information passes through to redesign to guide improvement efforts.

What about your environment? How does all this work with you?


Joe

Jinen said:

Joe,

"Begin with the end in mind" - well said. I agree that properly developed test instruments will perform the measurements.

Well, do you think analysis engine to understand the measurement of the class for a particular topic or subject as a whole is critical? Do you think such an indicator can assist the trainer to enhance his teaching methodology and deliver better results? Or do you think it will assist an instructional designer to understand gaps in the course design if any?

Thanks,
Jinen

[Jinen]

Joe,

Well, Like you said, in an intense hands on environment, you are left with very few options to conduct your tests leave alone analysis. I represent a software company and our intention is to solve exactly this particular pain area.

We currently solve this by allowing people to take a test on mobile phone. The results of these tests are later punched back into servers through WAP/SMS. Report generation can be done by the end user depending on his/her requirement.

To allow the trainer/lecturer to build a test, we offer an authoring tool - which can author tests for phones.
They can be distributed through bluetooth/WAP.

I would like to know, if this(our) solution can assist you in bridging the gap between your teaching and trainee's learning? What more can we do to make it more compelling?

Thanks,
Jinen

[Savannah]

I agree with testing, which provides the instructor an academic picture of each student and how they absorb, comprehend, and translate information. I have observed during classroom introductions adult learners are not always willing to admit up front they have deficiencies in a particular area or require more knowledge in regards to a program they should know.

I use testing in a small degree to gauge what techniques will be more beneficial to both students and me. This aids me to mentally check on the persons who will require extra time while providing for the students who are in-step with their progress.

All in all I believe concept testing helps to complete the full circle of training it provides a beginning and a successful ending for all.

[Gabriel Urrea]

Depending on the specific training it is my experience that rather than testing for concept knowledge I would test for the behavior application of the concept. I have created evaluations and feedback forms around concept comprehension and retention and found out that the real value is in evaluating the students ability to use the information in specific and relevant ways.

[Joseph McFadden]

Jinan,

Interesting concept! Since the application which we train is exclusivly oriented in non-mobile platforms, testing using mobile phones would not be appropriate or even credible. Testing on a different platform would not prove operational knowledge on the primary application.

I can see the crediblility of this type of testing in a learning organization where ad hoc testing whould capture a current "state-of" status of concept retention throughout the organization. Example: Members recently completed training on new electrical safety Lock-out/tag-out procedures. Mobile testing could query members on the standards ad hoc and offer insight on what needs to be emphisized during the next round of training.

Thanks

Joe

Jinen said:

Joe,

Well, Like you said, in an intense hands on environment, you are left with very few options to conduct your tests leave alone analysis. I represent a software company and our intention is to solve exactly this particular pain area.

We currently solve this by allowing people to take a test on mobile phone. The results of these tests are later punched back into servers through WAP/SMS. Report generation can be done by the end user depending on his/her requirement.

To allow the trainer/lecturer to build a test, we offer an authoring tool - which can author tests for phones.
They can be distributed through bluetooth/WAP.

I would like to know, if this(our) solution can assist you in bridging the gap between your teaching and trainee's learning? What more can we do to make it more compelling?

Thanks,
Jinen

[Joseph McFadden]

Hi Savannah,

Great twist! Are you suggesting pre-testing as a tool to tailor the presentation/training to the needs of your individual participants?

Joe
Savannah said:

I agree with testing, which provides the instructor an academic picture of each student and how they absorb, comprehend, and translate information. I have observed during classroom introductions adult learners are not always willing to admit up front they have deficiencies in a particular area or require more knowledge in regards to a program they should know.

I use testing in a small degree to gauge what techniques will be more beneficial to both students and me. This aids me to mentally check on the persons who will require extra time while providing for the students who are in-step with their progress.

All in all I believe concept testing helps to complete the full circle of training it provides a beginning and a successful ending for all.

[Joseph McFadden]

Cutting to the core issue! In most training situations "behaviour adjustment" is the objective again bringing back full circle the call for defined objectives as an early step in the development cycle. Gabriel, do you use objective based development / delivery? If so, how do (did) the objectives play into the development of your evaluation instruments?

Thanks
Joe

Gabriel Urrea said:

Depending on the specific training it is my experience that rather than testing for concept knowledge I would test for the behavior application of the concept. I have created evaluations and feedback forms around concept comprehension and retention and found out that the real value is in evaluating the students ability to use the information in specific and relevant ways.

[Jinen]

Savannah,

How do you perform concept testing in your environment?
Do you think mobiles can be used as a platform to conduct concept testing?

Thanks,
Jinen

Savannah said:

I agree with testing, which provides the instructor an academic picture of each student and how they absorb, comprehend, and translate information. I have observed during classroom introductions adult learners are not always willing to admit up front they have deficiencies in a particular area or require more knowledge in regards to a program they should know.
I use testing in a small degree to gauge what techniques will be more beneficial to both students and me. This aids me to mentally check on the persons who will require extra time while providing for the students who are in-step with their progress. All in all I believe concept testing helps to complete the full circle of training it provides a beginning and a successful ending for all.

[Jinen]

Just to add, on our website, www.deltecs.com is now offering a demo version of Drona - A concept testing tool that allows you to author tests for mobile phones and collate scores to measure performance at the backend.

I would invite you all to take a demo of the product and help me with your valuable feedback.

Thanks,
Jinen