In this post, Academic Technology Support Services (ATSS) staff members continue our exploration of generative AI tools that could support our work as instructional designers and academic technologists. Previously, we investigated the NOLEJ platform. This time, we looked at Bongo to investigate how it might be used in instructional design work to support teaching and learning.
Note: ATSS does not endorse the use of Bongo or any other tool that is not supported by the central Office of Information Technology. This blog post is part of an ongoing investigation of generative AI tools and their uses in teaching and learning.
Tool Overview
Bongo is a video-assignment platform that offers an AI Coach with features for instructors and students. Used in education and industry, Bongo allows students to practice and receive feedback on video assignments (e.g., video presentations or practice interviews). Additionally, Bongo’s AI Coach provides options to generate assignment learning objectives, offer auto analysis on video submissions, provide student feedback, and assist instructors with grading.
Bongo advertises several educational use cases:
- teacher licensure programs
- health coaching
- foreign language instruction
- skill demonstrations
- alternative to written assignments in any content area
Privacy + Security
When using any AI tool, it’s important to know how the tool protects the privacy and security of content-related and personal information. Bongo offers its Privacy Policy and Bongo AI Coach FAQ that addresses questions about information Bongo sends to OpenAI, how personally identifiable information (PII) is protected, and how Bongo AI Coach handles bias.
Accessibility
Bongo lists its accessibility features and VPAT on their Accessibility webpage. As Bongo is primarily a video-based tool, our testing verified that transcripts are automatically created for video assignment submissions (as part of the AI evaluation of student videos). Additionally, instructor and student videos can be captioned by 3 methods:
- Generate Automatically with an option to modify as necessary: this provides a good starting point for creating accurate captions to comply with ADA guidlines
- Import From File
- Enter Manually
Bongo in Canvas
While it is possible to integrate Bongo into Canvas, it can also be used as a stand-alone tool by either copying and sharing the assignment link or downloading a SCORM package. Using SCORM packages in Canvas can create significant maintenance and support issues. Make sure to connect with your local academic technology support staff to learn more.
Our Exploration Process
Bongo offers several assignment types– including Individual, Group, Question/Answer, Interactive Video – each with additional options such as peer review, self-reflection, and grading options. We will focus on the creation of an Individual video assignment using select Bongo AI Coach features.
Assignment Set-up
When setting up a Bongo assignment, it’s important to know how AI is integrated into the assignment itself. The Bongo AI Coach uses its Auto Analysis feature to provide feedback on the student’s Delivery (how the presentation is delivered) and Content (achievement of learning objectives, key terms, and phrases). Instructors can use Auto Analysis to assist with grading; students can use Auto Analysis to refine and revise their video assignments.
To use the Auto Analysis process, an instructor must first upload assignment Reference Materials – any text-based documents, transcripts, speaker notes, etc. Bongo analyzes the materials to generate assignment Learning Objectives. The objectives it generates are also the basis for AI-generated feedback provided to students or faculty.
We first uploaded a webinar transcript and learned that the uploaded Reference Materials cannot exceed 1500 characters; our webinar transcript was 21244 characters. To get around the character limit, we next uploaded the speaker notes extracted from the webinar’s Google Slides and deleted unnecessary text, bringing the character count to 1500.
The screenshot below illustrates the AI-generated Learning Objectives (and related criteria in the collapsed menu) that were generated from our Reference Materials:
During this step, we learned that instructors cannot upload their own learning objectives into the Bongo
assignment; they can edit the AI-generated objectives, though. Bongo’s rationale for this workflow is that their “proprietary process to analyze reference materials is designed to build Learning Objectives and criteria that can be effectively used in the feedback and scoring process. While edits are supported, creating Learning Objectives from scratch has been found to produce lower quality feedback and scoring results.”
Student Submission Process
Next, we tested the Bongo assignment from a student perspective. The video assignment could be recorded numerous times and each recording prompted AI-generated feedback that could be used to make improvements. Students could then select their best video to submit. The following three screenshots illustrate Bongo’s AI Coach-generated feedback to the student. Instructors will also see this feedback as part of the grading and evaluation process.
The first screenshot, shown below, illustrates AI Coach-generated feedback on the student’s Delivery (clarity, filler words, and speaking rate) and the automatically generated video transcript.
The second screenshot, shown below, illustrates AI Coach-generated feedback on student’s Tone (confidence, humor, formality, persuasiveness):
The final screenshot, shown below, illustrates AI Coach-generated Tips & Feedback that are based specifically on the assignment Learning Objectives. The AI Coach indicates Learning Objectives that were sufficiently met, partially met, or unmet. The feedback includes examples from the student’s video submission as a rationale for the achievement of each Learning Objective.
Assignment Grading and Evaluation
Instructors can choose to set up Smart Scoring, a feature that will automatically generate a score for students based on the Auto Analysis completed by Bongo AI. Instructors may choose to automatically use the Smart Scoring or to review and modify the score as they wish.
To test Smart Scoring we used the same uploaded resource materials and generated learning objectives as in the step above. Smart Scoring will generate a score for each student submission similar to the screenshot below. In addition to seeing a numeric score related to each learning objective, the Details section and Evaluation report provide additional insight into the scores generated.
Our test assignment prompt asked students to “list three new practices you learned about using the Canvas Gradebook.” However, Bongo had auto-generated five Learning Objectives for this assignment. Bongo AI looked for answers for all five learning objectives in the student submission. There didn’t seem to be a way to work around this, other than the instructor manually adjusting the scores while grading. Additionally, the Smart Scoring seemed derived from the learning objectives rather than the delivery proficiency (tone, clarity, fillers, etc).
Takeaways
This post does not address all Bongo assignment types or all of the available AI-features; however, our testing generated some considerations for tool use:
- Let students know that this type of AI use is new and encourage them to use a critical eye when considering the feedback they receive from the tool. Fostering this type of AI literacy is a good skill for students to develop.
- Some of the AI-generated feedback seemed straightforward and accurate, e.g., counting and identifying clarity, and filler words. The content-related analysis and evaluation, based on the AI-generated learning objectives, seemed more prone to error.
- Instructors who use the tool may gain some efficiencies in assignment creation but should carefully review and revise the AI-generated learning objectives and keywords for each assignment. Student feedback relies on this information.
- Use the AI-related grading feature with caution. There are inconsistencies in the process, e.g., the AI-generated evaluations are too broad, too specific, or not accurate.
- If uploading resource materials into Bongo, experiment with which type of content will yield the best results (learning objectives, keywords, feedback, and tips).