tutorials:ai_llm_tools
This is an old revision of the document!
Table of Contents
AI/LLM Tools
About
- A brief guide to using large language models (LLMs) often referred to as artificial intelligence (AI) tools, such as ChatGPT/Claude/Gemini, in the context of academic research and writing. By Chih-Horng Kuo (chk@gate.sinica.edu.tw). Suggestions are welcome.
- Target audience: undergraduate students, graduate students, and postdocs (in biology)
Preface
- For work: Ethics first!
- Respect the general rules of academic integrity
- As the user, you are responsible for the final output of your work
- Include proper usage statements
- For personal use:
- Emotional support can be valuable and beneficial; graduate school (or life) is hard!
- Should not be mistaken for true care or therapeutic depth
- Be careful of potential misuse and harm
LLMs Explained
What are LLMs?
- Key concept
- LLMs work by estimating probability relationships among tokens, not by indexing content like a database such as Wikipedia
- Key terms
- Token: the basic unit of text that LLMs process; typically equivalent to or smaller than a word
- Prompt: user input
- Session: a series of user input and LLM output
- Good practice: limit the scope and length of individual sessions
- Memory: information kept from previous interactions; some systems provide this function
- Artificial intelligence systems trained on large collections of text to generate human-like responses
- Human: form thoughts first, then use language to communicate those thoughts
- LLM: generate language by predict the next word (token) based on statistical patterns
- LLMs do not “think” or “understand” like humans do; the appearance of “thought” is a byproduct of those patterns
- LLMs mimic intelligence, but are not truly “intelligent”; can be very convincing!
Value
- Always-available conversation partners
- Often, just having a conversation helps you think much more clearly
- Particularly helpful in the context of emotional support
- In some ways, LLMs are like crystals of human knowledge
- Useful for learning new topics efficiently through interactive Q&A
- Perform especially well on topics with abundant, high-quality training data
- Highly proficient in language, particularly useful for clear and precise expression of ideas
- Great tools for self-improvement
- How much can LLMs be trusted? I don't know
- Have I successfully used LLMs to achieve deeper thinking and clearer writing? YES!
- What's the catch? Time and effort
Limitations
- Limitations and biases exist in both the training data and the model development process
- English is the dominant language
- Often optimized to be agreeable and non-confrontational
- Limitations of human knowledge and available text data sets
- LLMs cannot assess the quality of sources
- Cannot reliably distinguish between high- and low-quality content
- Cannot apply appropriate weighting
- Academia: “good” vs. “bad” papers
- Outside of academia: biased information from paid advertising and other influence
- LLMs do not know what they do not know; may make up false info (AI hallucination)
- For scientific writing
- Summarizing and explaining existing knowledge often work well
- May not be suitable for processing novel findings
Common Failure Modes
- Superficial content
- Oversimplification
- Overstatement
- False coherence
- Hallucination; fabricated web links, DOIs, and PubMed IDs
- Non-specific terminology for highly specific topics
Core Principles for Usage
- Use LLMs as tools to sharpen your thinking, not black boxes for quick answers
- Do not outsource your thinking process!
- Use LLMs in your thinking and learning: you become better
- Use LLMs to think for you: you skip the practice, bad for learning
- LLMs may not save time
- In fact, LLMs are most valuable when used to deepen the thinking process
- Explore possibilities, compare alternatives, and refine nuances all require time and effort, but the process and end product can be rewarding
- Example: You are deciding the title of your manuscript. You provided the abstract, do you ask LLMs to:
- (1) suggest a title, or
- (2) suggest three titles, compare the emphasis and tone, refine to express accurately, adjust based on the target audience
- LLMs act like mirrors; their responses reflect your input and framing
- Trained to be agreeable, they often conform to both your explicit instructions and implicit tone
- Helpful to make your implicit thoughts become explicit and better organized
Suggested practice
- Start with a solid frame of reference
- Provide clear context for the task, including your goals and expectations
- Learn the basic of markdown can be very helpful (“help me to learn the very basic of markdown language in 5 minutes, explain to me how can I use this in my prompt to improve the clarity of my instruction to you”)
- What exactly do you want to obtain?
- How do you want the LLM to act?
- Are your expectations realistic?
- Examples of possible uses (varying in feasibility and reliability):
- Polish arguments
- Organize thoughts
- Brainstorm new ideas
- Identify major gaps in reasoning
- Provide critical review
- Assess the output, make decisions, and provide feedback
- Use an iterative process; may involve gradual refinement or drastic changes
- Fact check, fact check, and fact check!!!
- Know when to stop; did the last iteration provide meaningful improvement?
- Exercises
- Use figure legends as input, ask for a manuscript outline as output; explore different ways of ordering the figures
- Use a poster abstract as input, ask for possible titles as output; compare how each emphasizes different aspects of the work or addresses different audiences
Human–AI Dynamics
- LLMs mirror your tone, assumptions, and style
- Output can become overly agreeable, especially in long sessions
- It is natural to feel defensive when an LLM points out flaws in your argument
- However, LLMs are easily swayed by your rebuttals
- Reinforcing your own biases is not useful
- Asking LLMs to role-play can be useful
- Not because the roles provide true expertise, but because the exercise can stimulate YOUR thinking
- For example, asking an LLM to act as a qualifying committee member or manuscript reviewer may help you prepare
- In these roles, LLMs can mimic tone and style, but they lack the knowledge, logic, and Judgment of human experts
Use Cases
- Force LLMs to criticize and challenge your thoughts
- Use the “Deep Research” function (available in both ChatGPT & Gemini) for targeted search of PubMed and follow-up summary
- Help to read dense papers
- Build custom databases of selected references using NotebookLM by Google
- Perform grammar check and copy editing
- Prepare for discussion with seminar speakers; info about you and the speaker as the input, suggestions of questions to ask as the output
- Draft or refine emails (with tone sensitivity)
- Obtain technology-related suggestions, such as server/intranet setup, hardware purchase planning, Linux/NAS configuration, or organization of computer files and directories
Practical Advice & Next Steps
- Test the tool with topics that you know well first for evaluation
- Push boundaries, learn the situations that LLMs are useful (or not)
- Different tools for different uses
- ChatGPT is chatty; highly conversational, discussions can take unexpected turns, great for exploration
- Claude is often more “get to the point” compares to ChatGPT
- Gemini integrates Google Search; useful for questions with clear answers and you want the web sources
- Learn to judge output, know when to reject the suggestions
- Over time, thoughtful use of LLMs can improve your writing, reflection, and decision-making skills
- AI/LLM tools are under intensive development, different tools may perform better at different tasks; develop your own taste and workflow, keep an eye on the development
- More advanced tools, such as Claude Cowork/Code, are not covered here, but definitely worth exploring
tutorials/ai_llm_tools.1778252972.txt.gz · Last modified: by chkuo