Highdigenous Fashion

wearable technology

How Kabakoo assesses AI learning support

By · · EN

Kabakoo distinguishes task validation from participation rewards and evaluates WhatsApp AI responses criterion by criterion. Completeness and conciseness have the lowest reported response-quality figures, while planned surveys will examine learners’ use of suggestions.

A table showing validation results: Rejected tasks (213) averaged 2 Kabakooins (min 1, max 3); Validated tasks (587) averaged 4 Kabakooins (min 3, max 5).
Kabakoo Academies

Assess tasks and recognize participation

At Kabakoo, a task does not have to be validated for a learner to earn Kabakooins. Task validation and participation rewards are distinct parts of its learning system. Alongside task assessment, Kabakoo evaluates WhatsApp AI responses, with completeness and conciseness recording the lowest figures in that evaluation.

Of 800 tasks evaluated by the AI Agent in the Kabakoo app, 73.38% (587 tasks) were validated and 26.63% (213 tasks) were rejected. These results describe evaluated tasks, rather than a count of individual learners.

Learners earn an average of 2 Kabakooins even when tasks are not validated, compared with an average of 4 Kabakooins for validated tasks. Exceptional submissions can earn up to 5 Kabakooins. Recognition therefore extends to attempts that do not receive validation.

Identify what learners ask for

Kabakoo also analyzes requests to its WhatsApp AI support service. Since its October 2024 launch, the service had handled over 26,000 support requests.

Of the user requests analyzed, over 79% concerned Kabakoo’s upskilling programs, while roughly 15% concerned platform navigation and technical issues. This analysis identifies the subjects of requests; the response evaluation examines the quality of the answers.

Evaluate responses criterion by criterion

Kabakoo assessed its WhatsApp AI-bot responses for relevance, accuracy, completeness, conciseness and coherence. In that evaluation, 69.4% received a top score for coherence, and 63.9% scored 5 for relevance.

Accuracy was at 61.7%, completeness at 51.7%, and conciseness at 34.5%. Completeness and conciseness were the lowest reported figures, identifying areas for improvement alongside the higher results for coherence and relevance. The evaluation reports these dimensions separately rather than combining them into a single response-quality score.

Extend assessment to learners’ use of suggestions

Following evaluation procedures discussed by Yang et al. (2024), Kabakoo plans to survey how often learners use the AI bot’s suggestions and how likely users are to recommend the system to a friend. These learner-use and recommendation measures remain planned, rather than completed results.

The assessment sits within Kabakoo’s wider approach of combining AI mentorship with community engagement, aiming to deliver consistent learning experiences to more people while retaining a personal touch. Follow Kabakoo’s news and updates for more on its work.

The Kabakoo newsletter

Research, field notes, and stories from Kabakoo in your inbox.