AI/LLM Tools

AI/LLM Tools

About

A brief guide to using large language models (LLMs) often referred to as artificial intelligence (AI) tools, such as ChatGPT by OpenAI and Gemini by Google, in the context of academic research and writing. By Chih-Horng Kuo (chk@gate.sinica.edu.tw). Suggestions are welcome.
Target audience: undergraduate students, graduate students, and postdocs (in biology)

Preface

For work: Ethics first!
- Respect the general rules of academic integrity
- As the user, you are responsible for the final output of your work
- Include proper usage statements
For personal use:
- Emotional support can be valuable and beneficial; graduate school (or life) is hard!
- Should not be mistaken for true care or therapeutic depth
- Be careful of potential misuse and harm

LLMs Explained

What are LLMs?

Key concept
- LLMs work by estimating probability relationships among tokens, not by indexing content like a database such as Wikipedia
Key terms
- Token: the basic unit of text that LLMs process; typically equivalent to or smaller than a word
- Prompt: user input
- Session: a series of user input and LLM output
  - Good practice: limit the scope and length of individual sessions
- Memory: information kept from previous interactions; some systems provide this function
Artificial intelligence systems trained on large collections of text to generate human-like responses
- Human: form thoughts first, then use language to communicate those thoughts
- LLM: generate language by predict the next word (token) based on statistical patterns
- LLMs do not “think” or “understand” like humans do; the appearance of “thought” is a byproduct of those patterns
- LLMs mimic intelligence, but are not truly “intelligent”; can be very convincing!

Value

Always-available conversation partners
- Often, just having a conversation helps you think much more clearly
- Particularly helpful in the context of emotional support
In some ways, LLMs are like crystals of human knowledge
Useful for learning new topics efficiently through interactive Q&A
Perform especially well on topics with abundant, high-quality training data
Highly proficient in language, particularly useful for clear and precise expression of ideas
Great tools for self-improvement
- How much can LLMs be trusted? I don't know
- Have I successfully used LLMs to achieve deeper thinking and clearer writing? YES!
- What's the catch? Time and effort

Limitations

Limitations and biases exist in both the training data and the model development process
- English is the dominant language
- Often optimized to be agreeable and non-confrontational
- Limitations of human knowledge and available text data sets
LLMs cannot assess the quality of sources
- Cannot reliably distinguish between high- and low-quality content
- Cannot apply appropriate weighting
- Academia: “good” vs. “bad” papers
- Outside of academia: biased information from paid advertising and other influence
LLMs do not know what they do not know; may make up false info (AI hallucination)
For scientific writing
- Summarizing and explaining existing knowledge often work well
- May not be suitable for processing novel findings

Common Failure Modes

Superficial content
Oversimplification
Overstatement
False coherence
Hallucination; fabricated web links, DOIs, and PubMed IDs
Non-specific terminology for highly specific topics

Core Principles for Usage

Use LLMs as tools to sharpen your thinking, not black boxes for quick answers
Do not outsource your thinking process!
- Use LLMs in your thinking and learning: you become better
- Use LLMs to think for you: you skip the practice, bad for learning
LLMs may not save time
- In fact, LLMs are most valuable when used to deepen the thinking process
- Explore possibilities, compare alternatives, and refine nuances all require time and effort, but the process and end product can be rewarding
- Example: You are deciding the title of your manuscript. You provided the abstract, do you ask LLMs to:
  - (1) suggest a title, or
  - (2) suggest three titles, compare the emphasis and tone, refine to express accurately, adjust based on the target audience
LLMs act like mirrors; their responses reflect your input and framing
- Trained to be agreeable, they often conform to both your explicit instructions and implicit tone
- Helpful to make your implicit thoughts become explicit and better organized
Suggested practice
- Start with a solid frame of reference
  - Provide clear context for the task, including your goals and expectations
  - What exactly do you want to obtain?
  - How do you want the LLM to act?
  - Are your expectations realistic?
  - Examples of possible uses (varying in feasibility and reliability):
    - Polish arguments
    - Organize thoughts
    - Brainstorm new ideas
    - Identify major gaps in reasoning
    - Provide critical review
- Assess the output, make decisions, and provide feedback
- Use an iterative process; may involve gradual refinement or drastic changes
- Fact check, fact check, and fact check!!!
- Know when to stop; did the last iteration provide meaningful improvement?

Human–AI Dynamics

LLMs mirror your tone, assumptions, and style
Output can become overly agreeable, especially in long sessions
- It is natural to feel defensive when an LLM points out flaws in your argument
- However, LLMs are easily swayed by your rebuttals
- Reinforcing your own biases is not useful
Asking LLMs to role-play can be useful
- Not because the roles provide true expertise, but because the exercise can stimulate YOUR thinking
- For example, asking an LLM to act as a qualifying committee member or manuscript reviewer may help you prepare
- In these roles, LLMs can mimic tone and style, but they lack the knowledge, logic, and Judgment of human experts

Use Cases

Force LLMs to criticize and challenge your thoughts
Use the “Deep Research” function (available in both ChatGPT & Gemini) for targeted search of PubMed and follow-up summary
Help to read dense papers
Build custom databases of selected references using NotebookLM by Google
Perform grammar check and copy editing
Prepare for discussion with seminar speakers; info about you and the speaker as the input, suggestions of questions to ask as the output
Draft or refine emails (with tone sensitivity)
Obtain technology-related suggestions, such as server/intranet setup, hardware purchase planning, Linux/NAS configuration, or organization of computer files and directories

Additional Points

Test the tool with topics that you know well first for evaluation
Push boundaries, learn the situations that LLMs are useful (or not)
Different tools for different uses
- ChatGPT is chatty; highly conversational, discussions can take unexpected turns, great for exploration
- Gemini integrates Google Search; useful for questions with clear answers and you want the web sources
Learn to judge output, know when to reject the suggestions
Over time, thoughtful use of LLMs can improve your writing, reflection, and decision-making skills
AI/LLM tools are under intensive development, different tools may perform better at different tasks; develop your own taste and workflow, keep an eye on the development

Table of Contents

AI/LLM Tools

About

Preface

LLMs Explained

What are LLMs?

Value

Limitations

Common Failure Modes

Core Principles for Usage

Human–AI Dynamics

Use Cases

Additional Points