LLMPapers

Repo for summary of papers on Large Language Models (LLMs)

“Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow” by Maria del Rio-Chanona, Nadzeya Laurentsyeva, and Johannes Wachs1

• The paper investigates how the release of ChatGPT, a large language model that can answer questions on various topics, affected the activity on Stack Overflow, a popular online platform where computer programmers ask and answer questions. The paper compares the activity on Stack Overflow with similar platforms for mathematics and other languages, where ChatGPT is less capable or accessible. The paper finds that after the release of ChatGPT, the activity on Stack Overflow decreased by 16%, and this effect increased over time and was larger for more popular programming languages. The paper argues that this implies that more users are using ChatGPT as a substitute for Stack Overflow, and that this reduces the amount of public knowledge available on the web. The paper also discusses the implications of this trend for future research and learning, both for humans and machines.

Welcome to the era of ChatGPT et al.

• This paper gives a high level overview of emergence of large language models, their pitfalls like hallucinations, their limitations in certain feilds like mathematics and how these large language models will change the economy of jobs in coming years. The paper talks about RLHF which uses human feedback to improve the responses of a large language model. The authors raise relevant points like who owns the content when the humans and the chat bots co-author the content. The authors also touch upon the importance of prompt engineering and ability of humans to learn how to craft effective prompts which allows these large language models to generated required responses.

LLMs for Bad Content Detection: Pros and Cons

• The article titled "LLMs for Bad Content Detection: Pros and Cons" discusses the methods for identifying and detecting harmful content on the Internet. It evaluates two main approaches: training supervised classifiers and using large language models (LLMs) for this purpose and provides advantages and disadvantages of either approach.

What the Heck is RAG? And How to Build It

• This article describes the RAG scenario in context of LLMs. It talks about how you could use a LLM to answer questions about a certain document. The author also provides a sample code snippet to build a simple RAG model using python's machine learning APIs.