A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians
arxiv.org
We interviewed twenty professional comedians who perform live shows in front of audiences and who use artificial intelligence in their artistic process as part of 3-hour workshops on ``AI x Comedy'' conducted at the Edinburgh Festival Fringe in August 2023 and online. The workshop consisted of a comedy writing session with large language models (LLMs), a human-computer interaction questionnaire to assess the Creativity Support Index of AI as a writing tool, and a focus group interrogating the comedians' motivations for and processes of using AI, as well as their ethical concerns about bias, censorship and copyright. Participants noted that existing moderation strategies used in safety filtering and instruction-tuned LLMs reinforced hegemonic viewpoints by erasing minority groups and their perspectives, and qualified this as a form of censorship. At the same time, most participants felt the LLMs did not succeed as a creativity support tool, by producing bland and biased comedy tropes, akin to ``cruise ship comedy material from the 1950s, but a bit less racist''. Our work extends scholarship about the subtle difference between, one the one hand, harmful speech, and on the other hand, ``offensive'' language as a practice of resistance, satire and ``punching up''. We also interrogate the global value alignment behind such language models, and discuss the importance of community-based value alignment and data ownership to build AI tools that better suit artists' needs.
  • self@awful.systemsEnglish
    60·
    4 months ago

    At the same time, most participants felt the LLMs did not succeed as a creativity support tool, by producing bland and biased comedy tropes, akin to ``cruise ship comedy material from the 1950s, but a bit less racist’'.

    holy shit that’s a direct quote from the paper

    • bitofhope@awful.systemsEnglish
      22·
      4 months ago

      The phrasing “a bit less racist” suggests a nonzero level of racism in the output, yet the participants also complain about the censorship making the bot refuse to discuss sensitive topics. Sounds like these LLMs can only be boringly racist.

  • Amoeba_Girl@awful.systemsEnglish
    37·
    4 months ago

    Spam machines are only ever funny or interesting by accident. The more they smooth out the wrinkles the more creatively useless they become. The tension is sort of fascinating.

    Like I’ve always been interested in generative poetry and other manglings of text, and ChatGPT’s so fucking dull compared to putting a sentence through babelfish a few times.

    • hrrrngh@awful.systemsEnglish
      6·
      4 months ago

      Honestly, I’ve gotten more laughs out of messing with markov chains with my friends than anything ChatGPT could put out

    • 200fifty@awful.systemsEnglish
      5·
      4 months ago

      Before the big AI boom, I actually did a project where I used inferkit to generate text for the comedy factor because the unhinged nightmare garbage it spit out was extremely entertaining. I just can’t imagine using chat gpt in the same way, it’s so boring

  • jordanlund@lemmy.worldEnglish
    37·
    4 months ago

    Read through the paper looking for sample jokes, found none. :(

    But this was the issue with the George Carlin bit that was online. I listened to it, it was a reasonable approximation of George’s voice and intonation.

    But when it got to the part where it said “I think we can all agree, there’s one comedian better off as AI… Bill Cosby.” and I went “OK, AI did not write that.” AI doesn’t get subversion.

    Turns out…

    https://arstechnica.com/ai/2024/01/george-carlins-heirs-sue-comedy-podcast-over-ai-generated-impression/

  • David Gerard@awful.systemsOPMEnglish
    26·
    4 months ago

    oh, and these twenty comedians were using LLMs for writing already. They didn’t want their names revealed, for some reason.

    • deborah@awful.systemsEnglish
      22·
      4 months ago

      The adverse impacts section was just the comedians saying “we’ve already lost friends, everyone hates us” but the conclusion was “here’s how comedians should use our tool.”

  • swlabr@awful.systemsEnglish
    25·
    4 months ago

    I can imagine a comedian using an LLM to check if a joke or punchline has been done before, but that would require the LLM to actually work and give accurate information. Also if you are a comedian using an LLM, you probably don’t actually care about whether or not you are plagiarising someone, so I guess this is all moot.