Design Document

Formative Evaluation

 

The formative evaluation that took place was during the 11th week of the project. The Digital Badge site was in it's early stages. The evaluations that took place were between one teacher as a content area expert and one subject matter expert of the interface. A field trial also took place with a two teachers from the Technology Cadre.

 

Three of the evaluations took place as one-to-one evaluations with and one took place online due to a conflict in scheduling. The self paced evalution took place via email by sending the link to the digital badge site. All of the videos and forms are embedded within the site and the evaluator did not need additonal files or links to complete the evaluation. 

 

One-to-One Evaluations

The instructional materials that were used in the one-to-one evaluations were in the early stages of development. The evaluations took place at this time to get a maximized feedback from the end users. Changes took place soon after the evaluators discussed their feedback with me and understood the process of the digital badge cumulative collection. 

 

Subject Matter Exper Evaluation

 

1. What was the purpose of the evaluation session? The purpose was to evaluate the overall website design, the fluidity of the modules and the navigation menu. 

 

2. Who was involved in the session? What is his/her profile? A technology teacher who has experience in web design agreed to evaluate the website. 

 

3. Where did the session take place? The evaluation took place in her home office where she has access to wifi and a quiet space to work. 

 

4. What were the results? The results of this evaluation from the subject matter expert in regards to the interface were as follows:

 

Clarity

She reported that the site was a little bit difficult to follow because the training site was within another site. She suggested that I extract the site and make it independent of anything else. She did say that the it was visually appealling and that the elements meshed together well. 

 

Impact

She would like to see the extracted site and see how the fluidity of the site would appear after the changes were made. 

 

Feasibility

She asked what did she need to do to take advantage of the incentives being offered on the Digital Badge site. She was willing to transfer school districts. 

 

5. What was your reaction to the results? I agreed with the evaluation and mentioned to her that my reasons for keeping both sites together was to not take visitors to my website to other pages. I do think that this site needs to be stand alone. 

 

6. What was your action to the results? I extracted the Digital Badge training pages and made them into it's own website. I also changed the navigation menu and simplified it.

 

Learner Evaluation

 

1. What was the purpose of the evaluation? The purpose of the learner evaluation was to evaluate the feasibility of use such as the directions on where to start and where to go next. The engagement of the website was the second area on what makes a user want to stay on the website. The final portion was on the content, is the content applicable to your instructional area.

 

2. Who was involved in the session? What is his/her profile? The learner evaluation was completed by one content area expert who currently integrates many types of technologies into high school instruction.

 

3. Where did the session take place? The content area expert completed the evaluation on her own time because of a scheduling conflict. She completed the evaluation at home.

 

4. What were the results? Please note that changes had been made following the Subject Matter Expert's evaluation. The results in regards to the learner evaluation were as follows:

 

Clarity

The evaluator found the directions on the training site to be easy to understand. She stated that the introductory video was very engaging and she couldn't wait to claim a badge. 

 

Impact

As she navigated to the What's Next section she realized that there were incentives behind the digital badges. She was pleasantly surprised with that section. 

 

Feasibility

The evaluator appreciated that there were no pop-ups and that everything was included in the website. She was also curious to know more about the forms and how they were embedded within the website. I mentioned that Google Forms would be one of the badges that she could complete a self-paced training on in the future.  

 

Her recommendations:

How soon will the site be available?

Can you include an "Other" option because there are some apps that I use that are not listed.

Will I have to re-claim badges next school year?

 

5. What was your reactions to the results?

I was relieved that she was excited about the process of claiming badges. She did bring up a valid point that I had not thought of. "Will I have to re-claim badges next school year?" 

 

6. What was your action to the results?

I added the "Other" option to the forms and I am going to evaluate the question about claiming badges in consecutive years. 

 

Field Trial

 

The field trial was conducted with 2 of the 10 Technology Cadre teachers. The plan was to have the teachers complete the field trial before the holiday break. The results of the field trial are listed below.

 

1. What was the purpose of the evaluation? The purpose of this evaluation is to find out if there are additional changes that need to be made to the training site. 

 

2. Who was involved in the session? What is his/her profile?  The two evaluaotrs are teachers that form part of the Technology Cadre. One teaches at the elementary campus and the other teaches in the middle school campus. 

 

3. Where did the session take place? The two teachers completed the evaluation after school hours in their classrooms.

 

4. What were the results? The teachers mentioned that although they were exhausted at the end of the day they were interested in the digital badge site. The found the site to be easy to navigate and one mentioned that the volume was really high on her speakers and when the video started it startled her.

 

5. What was your reaction to the results? The feedback that was received was encouraging. I know that the majority of the teachers will be using the training site after hours and it's encouraging that the site is motivational enough to engage them to claim badges or complete trainings on their own time.

 

6. What was your action to the results? I will be adding a message that the audio may be loud if their speakers are on too high.