Return to homepage

Topic

Responsible Metrics and Algorithm

It is important to distinguish upfront between metrics, which refers to what is being measured, and algorithms, which refers to how it is being measured. Metrics should be controlled by academic institutions, and deliberately chosen, rather than relying on those sold by commercial vendors. Algorithms, do not necessarily need to be controlled by each academic institution, but must be carefully understood and monitored.

Contents

  • Emerging Challenges: Rising Privacy and Surveillance Concerns

    The rise in commercial publishers usage of tracking software in services sold to academic libraries which allows them to collect and sell data to third parties, as well as the risks and inequities of online exam proctoring tools, require attention of the academic community.

  • Continuing Challenges: Conflicts of Interest

    Commercial publishers are pursuing interests that put them at odds with the interests of the academic community and tolerate internal conflicts of interest. Unfortunately, the responsibility for highlighting and resolving conflicts of interest falls squarely onto the academic community.

  • Community Actions

    Establish Inclusive Governance Structures

    It is vital for the governing bodies of infrastructure services to include representation from the communities they serve in order to ensure that management stays accountable to the community’s evolving needs.

  • Strategic Choices

    Quantitative vs. Qualitative Metrics

    While institutions may not be ready to abandon the usage of quantitative metrics to evaluate their faculty, they should consider engaging in a genuine debate on the relative weight that they place on quantitative vs. qualitative assessment.

  • Strategic Choices

    Algorithms vs. Humans

    It is only a matter of time before artificial intelligence further pervades campus decision-making in ways that impact equity, privacy, and allocation of resources.

  • What Do We Mean By Data And Data Infrastructure

    We talk about two types of data. The first is Research Data, which refers to the data academic institutions generate through their research activities. The second is Grey Data, which refers to the vast amount of data produced by universities outside of core research activities.

  • Education Companies

    The Products

    Anecdotal evidence suggests that these systems built and maintained by publishers capture massive amounts of data about student and faculty behavior that go beyond what is necessary for accomplishing their core objectives (i.e. improving student outcomes). Institutions, faculty and students should think about the accumulation and use of data collected and retained by schools and commercial vendors.