2/11/22: VW videos and discussion

   Agenda & minutes:

• Welcome to the 5th meeting! 
• Any updates/news/inputs/comments?
• VW getting ready for candidacy exam.
• Organizational issues
• This agenda and minutes is on a blog, https://AIntwhatitusedtobe.blogspot.com. You can add comments if you like, just like for many other blogs.
• This meeting will be automatically ended by Zoom after 40 min. 
• Future meetings: 
• Feb. 18 meeting: We will focus on "Attention is All You Need" and read and discuss it together.
• Feb. 25 meeting: Any ideas/requests?
• VW presented his 1-minute videos and accompanying discussion: 
• https://www.youtube.com/watch?v=6Mu9sv9B4W0&ab_channel=ChrisFix;  and 
• https://www.youtube.com/watch?v=JWPzdtwO2HI
• Readings and etc. that have been suggested:
• "Attention is all you need," https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
• We read the first sentence of the abstract and discussed it quite a bit. This is a classic and important paper. We will devote a future meeting to reading through more of it, bit by bit. 

•  Attention is all you need, A Vaswani, N Shazeer, N Parmar… - Advances in neural …, 2017 - proceedings.neurips.cc … attention-weighted positions, an effect we counteract with Multi-Head Attention as described in section 3.2. Self-attention, sometimes called intra-attention is an attention … In Table 3 rows (A), we vary the number of attention heads and the attention key and value dimensions,… Cited by 35980 Related articles All 35 versions

• MM suggests explainable AI as a reading/discussion topic.
• Anticipative Video Transformer, https://facebookresearch.github.io/AVT/?fbclid=IwAR1RurSM33v8baN10H9JCX_dvVNtscydsLupaB8NMgKOmNIPjIwD3XO2vOA.
• "Deep learning—a first meta-survey of selected reviews across scientific disciplines, their commonalities, challenges and research impact," https://peerj.com/articles/cs-773. 
• We read the abstract last time and scanned more this time. It is not clear whether we should continue reading material from it. Any opinions/thoughts/comments?
• Some quantum computing references we could read as needed (from VW):
• - Quantum crossing threshold (free): https://www.nature.com/articles/s41586-021-04273-w
  - Crossing threshold in silicon: https://www.nature.com/articles/s41586-021-04182-y
  - Three-qubit donor processor in Si: https://www.nature.com/articles/s41586-021-04292-7