ISA Tool - User Tests in the Context of GLAMhack 2022

Documents / Links:

• “Image to Concept” Project
• Development Version of the ISA-Tool
• Commons: Depiction Guidelines

Usability Tests could focus on:

• Look, feel, and usability of the tagging tool with the machine vision / metadata-to-concept enhancements in place.
• Usefulness of the tool from the point of view of the Wikimedia Commons community (alignment with community norms, helpfulness in getting the ‘work’ done, etc.)
• Usefulness of the structured data that is being generated thanks to the tool from the perspective of GLAM institutions (e.g. in the context of content donations).

The involvement of GLAM staff in the project would be most helpful with regard to 1 and 3.

• The level of involvement will depend on their availability, readiness, personal background, prior experience, organizational context, etc. – Forms of involvement may include:

• “Play” around with the ISA Tool, make a few edits, and give us feedback from the point of view of an occasional user (with or without prior experience as a contributor to Wikimedia Commons).
• Make a substantial contribution to the tagging of a collection / campaign (several hours of work with the tool, possibly exchanges with the community), and give us feedback from the point of view of an advanced user of the tool. Assess to what extent they could imagine that someone at their institution would use the tool for the tagging of their own image collections.
• Assess the usefulness of the structured data generated with the help of the tool. If possible: Compare it to other tools / other approaches which they are familiar with.
• Help us identify / provide data that could be used for benchmarking the quality of the tagging on Wikimedia Commons with the help of the ISA Tool against other algorithm-supported tagging tools (automatic or semi-automatic). – This would require that the given institution already has used such approaches on their own image collections that are available on Wikimedia Commons.  
• Evaluate the desirability of using the ISA Tool as the basis for their own crowdsourcing campaigns. – This would require that the given institution already has carried out or at least given some thought to carrying out their own crowdsourcing campaigns.

User stories / usage scenarios