Short tips for SIGIR

Peter Bailey @peter_r_bailey

#sigir2013 A series of short tips from reviewing papers for SIGIR over the years.

Peter Bailey @peter_r_bailey

#sigir2013. Tip 1. Titles describe the paper; don't mislead, overpromise, or be unduly punny. Less is more. A tweet's length is too long.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 2. Abstracts summarize the paper. In 1 short paragraph. Subject. Motivation. Goal. Method. Achievement. Make me want to read.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 3. Intro: What are you doing? Why is it interesting and novel? Will I learn something in the next hour? 1/2 to 3/4 page max.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 4. Related Work. 30-40 citations is sweet spot; less means you don't know the area. Don't omit key works from other people.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 5. Report experimental results with appropriate statistical info like error bars, confidence intervals, effect size. Please.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 6. ML is a means to an end, not the main meal. Feature engineering is only interesting if it gives insights on user behavior.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 7. Use Greek letters and maths/stats for precision and brevity. Explain intuitions in English. Don't assume familiarity.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 8. Unless space-constrained, put lead author's name before citation cross-ref, so I don't have to flip back and forth always.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 9. Citations. Run cross-ref update in Latex or Word before final submission. Check author names' spelling; they might review!

Peter Bailey @peter_r_bailey

#sigir2013 Tip 10. If evaluating with a test collection, use more than 1. Don't use pre 2000 exclusively. There's more to life than ad hoc.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 11. Working with big log data doesn't make it interesting. Insights make it interesting. Especially ones which generalize.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 12. Building real IR systems involves much compromise. Don't overclaim for your new algorithm unless you've already tried it.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 13. What's product interesting is not always research interesting, and vice versa. What's both is not always publishable.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 14. Graphics. Make legends readable when printed on paper. If color essential to interpret output, make note in fig caption.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 15. When selecting data to exclude, describe what you did and why. Otherwise it's not clear what the bias in dataset will be.

Peter Bailey @peter_r_bailey

#sigir2013 Tip16. Negative results can be interesting. What didn't work, and why not? Surely not all expts led directly to success? #dreamon

Peter Bailey @peter_r_bailey

#sigir2013 Tip 17. Your first audience is overworked, on a deadline, tired, had a glass of wine and put the kids to bed. That is your prior.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 18. Two words: Strunk and White. Read then follow: Rules of usage. Principles of composition. Matters of form. Misused words.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 19. Don't pad out with lower quality work. >4:5 papers are rejected. Many great 8 page papers in past. Make it all shine.

Peter Bailey @peter_r_bailey

#sigir2013 Tip 20. Make your contributions clear. Both at the start and end of the paper. Also, Conclusions are not a summary. What changes?

Peter Bailey @peter_r_bailey

#sigir2013 Tip 21. Co-author with people smarter than you. You'll learn from them. They likely have skills you don't have. Best, it's fun!

Peter Bailey @peter_r_bailey

#sigir2013 Tip 22. Acknowledge those who helped you out. Thanks cost little, while forgetting may offend. If big help, make them a co-author

Peter Bailey @peter_r_bailey

#sigir2013 Tips <end/> Thanks for all the feedback, hope you enjoyed them. What did I miss? What do you most love/hate in SIGIR papers?

Peter Bailey @peter_r_bailey

#sigir2013 Tips <thanks> Big shout out to many colleagues over years, especially to Dave Hawking, Nick Craswell, Ryen White & Susan Dumais.