(Day 85) How to write a great research paper

Ivan Ivanov · March 26, 2024

theory   reading-research

Hello :) Today is Day 85!

A quick summary of today:

  • An amazing talk about how to write a great research paper by Simon Peyton Jones
  • Submitted a draft for a research proposal for next semester

I actually saw this talk after, writing the short proposal, but still it kind of motivated me for next semester.

Tips on how to write a great research paper by Simon Peyton Jones (talk)

  1. Dont wait, write!

  • This forces is to be clear, focused
  • Crystallises what we don’t understand
  • Opens the way to dialogue with others: reality check, critique, and collaboration
  • Writing is the way to develop your idea
  1. Identify your key idea!
  • Your goal: to convey a useful and re-usable idea
  • You want to infect the mind of your reader with your idea, like a virus
  • Papaers are far more durable than programs
  • Do not be intimidated
    • Fallacy: you need to have a fantastic idea before you can write a paper
    • Don’t wait to have a good idea. Write stuff about any idea
    • We only discover ideas are good later
  • The idea
    • Your paper should have just one ‘ping’: one clear, sharp idea
    • You may not know exactly what the ping is when you start writing; but you must know when you finish
    • If you have lots of ideas, write lots of papers
  • Can you hear the ‘ping’ ?
    • Many papers contain good ideas, but do not distil what they are
    • Make certain that the reader is no doubt what the idea is. Be 100% explicit!
      • ‘The main idea of this paper is …’
      • ‘In this section we present the main contributions of the paper …’
  1. Tell a story
  • Imagine you are explaning at a whiteboard
    • Here is a problem
    • It’s an interesting problem
    • It’s an unsolved problem
    • Here is my idea
    • My idea works (details, data)
    • Here’s how my idea compares to other people’s approaches
  • Structure
    • Title (1000 readers)
    • Abstract (4 sentences, 100 readers)
    • Introduction (1 page, 100 readers)
    • The problem (1 page, 10 readers)
    • My idea (2 pages, 10 readers)
    • The details (5 pages, 3 readers)
    • Related work (1-2 pages, 10 readers)
    • Conclusion and further work (0.5 pages)
  1. Nail your contributions to the mast
  • The introduction (1 page)
    • Describe the problem
      • Consider molehills not mountains, for problems to solve
      • Use an example to introduce the problem

  • State your contributions
    • Write the list of contributions first
    • The list of controbutions drives the entire paper: the paper substantiates the claims you have made
    • Reader thinks ‘gosh, if they can really deliver this, that’s amazing; I’d better read on’
    • Clear, bulleted list of contributions
    • Contributions should be refutable

  • No ‘rest of this paper is…’
  • Instead, use forward references from the narrative in the introduction. THe introduction (including contributions) should survey the whole paper, and therefore forward reference every important part
  1. Related work: later
  • No related work yet!

  • Problem 1: the reader knows nothing about the problem yet; so your (highly compressed) description of various technical tradeoffs is absolutely incomprehensible
  • Problem 2: describing alternative approaches gets between the reader and your idea
  • Credit
    • Fallacy: to make my work look good, I have to make other people’s work look bad
    • The truth: credit is not like money
      • Warmly aknowledge people who have helped you
      • Be generous to the competition
      • Aknowledge weaknesses in your approach
      • Giving credit to others does not diminish the credit you get from your paper!
  1. Put your readers first
  • Structure
    • The idea: Consider a bifircuated semi-lattice D, over a hyper-modulated signature S. Suppose pi is an element of D. Then we know for every such pi there is an epi-modulus j, such that pj < pi.
    • Sounds impressive, but
    • Sends the readers to sleep, and/or makes them feel stupid
    • When writing, don’t get stuck explaining something that is not worth reading. If there is an obvious/popular path that was not taken, explain why to the reader.
  • Presenting the idea
    • Explain it as if you were speaking to someone using a whiteboard
    • Conveying the intuition is primary, not secondary
    • Once your reader has the intuition, they can follow the details (but not vice versa)
    • Even if they skip the details, they still take away something valuable
  • Conveying the intuition
    • Introduce the problem, and your idea, using EXAMPLES and only then present the general case
    • General -> specific
  • Do not recapitulate your personal journey of discovery. This route may be soaked with your blood, but that is not interesting to the reader
  • Instead, choose the most direct route to the idea
  1. Listen to your readers
  • Getting help
    • Experts are good
    • Non-experts are also very good
    • Each reader can only read your paper for the first time once!
    • Explain carefully what you want (‘I got lost here’ is much more important than ‘Pythn is mis-spelt’)
    • Get your paper read by as many friendly guinea pigs as possible
  • Getting expert help
    • A good plan: when you think you are done, send the draft to the competition saying ‘could you help me ensure that I describe your work fairly?’
    • Often they will respond with helpful critique
    • They are likely to be your referees anyway, so getting their comments or criticism up fron is jolly good
  • Listening to your reviewers
    • Treat every review like the hold dust - be (truly) grateful for criticism as well as praise
    • This is really, really, really hard, but really, really, really, really, really, really, really, important
    • Read every criticism as a positive suggestion for something you could explain more clearly
    • DO NOT respond ‘you stupid person, I meant X’
    • INSTEAD, fix the paper so that X is apparent even to the stupidest reader
    • Thank them warmly. They have given up their time for you

Now about the proposal ~

It is mostly based on the prompt engineering literature review I wrote last semester, submitted and presented at the Korea Conference on Software Engineering 2024. The topic name I wrote is Analyzing ChatGPT’s Financial Statements Understanding Using Different Prompt Engineering Strategies. The idea is basically to analyze ChatGPT’s financial statements understanding using different prompt engineering strategies. The ‘beta’ plan I submitted is to do a literature review prompting strategies used for ChatGPT, along with an examination of papers regarding financial statements Q&A and stakeholders’ information preferences. Subsequently gather some of the financial statements referenced in popular literature alongside stakeholders’ questions and points of interest. Use popular evaluation metrics such as ROUGE for summarization, and precision, recall and accuracy for answers. It is quite basic but I am excited to start doing some kind of research.

Also, when I went to my professor to talk about it, he asked me whether I know about langchain, fine-tuning and my general knowledge of machine learning. I was so excited when I said yes because that is what I have been studying these so far 85 days, but the best thing is, is that there is so much more to learn, a lot to practice and test my knowledge.

That is all for today!

See you tomorrow :)