data subtldr week 10 year 2025

The Reddit thread titled How true is this? has received mixed responses, with some comments expressing criticism and others providing more measured perspectives. The thread largely discusses the roles of webbing engineer, data scientist, and machine learning (ML) engineer, with many comments pointing out the differences between these roles and critiquing the original post's oversimplification. A few comments also call out the post for being cringe-worthy and making inappropriate comparisons. One insightful comment emphasizes that the need for a specialized ML engineer role depends on the size of the enterprise and the number of ML use cases it has, suggesting that a data engineer with sufficient ML knowledge can be more efficient in smaller settings.

The Reddit thread titled r/dataengineering roasted by ChatGPT has sparked lively discussion with 95 comments. Users are expressing feelings of surprise and humor at the bot's ability to roast them with comments like Enjoy your life as a glorified data janitor. There's also acknowledgment of the reality of the industry, with users pointing out the challenges faced by aspiring data analysts and scientists, and the relatability of phrases like 4 CPUs and a dream. Jokes about the bot's potential future roles, such as debugging pipelines, show a light-hearted acceptance of AI's increasing prominence. Overall, the thread reflects a mix of self-deprecating humor and serious commentary about the data engineering field.

The Reddit thread is about a user who successfully passed a job interview by leveraging his SQL skills and the importance of taking detailed notes. The user shared his interview experience and knowledge on SQL, CTEs, Window Functions, and Regex. Top comments include a detailed learning path from a user aiming to become a Data Scientist, users asking for the course details, and several professionals sharing their experiences with SQL, Regex, and window functions. While some users admitted to not knowing certain SQL concepts, others offered explanations and shared their daily use of these tools. There were also several invitations for peer learning and collaboration. The overall sentiment was positive and encouraging, with users learning from each other and sharing resources.

Andrew Barto and Richard Sutton have been awarded the 2024 ACM A.M. Turing Award for their foundational work in reinforcement learning. Reddit users celebrated the announcement, praising Sutton for his approachability and visionary perspective on AI. They also highlighted the importance of sustained, long-term basic research in this field, pointing out that the major breakthroughs we see today are the result of 40 years of continuous academic research. Some comments humorously anticipated reactions from AI researcher Schmidhuber. The thread also acknowledged Canada as a significant contributor to modern AI.

The Reddit thread discusses a solution named CompressARC, which achieves 34.75% on the training set and 20% on the evaluation set for ARC-AGI. The solution trains a small neural network from scratch for each puzzle, without pretraining or a dataset, using just gradient descent. The comments highlight the impressive write-up, intuitive figures, and transparent approach of the authors. Some users found the blog post complex and intellectually stimulating, while others show interest in its potential performance on different datasets. There's also a discussion about this Just-In-Time training becoming more common in AI. The overall sentiment is positive, acknowledging the originality and complexity of the work.

The Reddit thread titled Dear SQL, just pivot my damn table has a mixed sentiment with 50 comments. Many users express the need for SQL to have dynamic pivot columns, which would automatically include new attributes as columns without manual input. There are suggestions to select column_name from information_schema.columns for dynamic pivot construction. Some users mention Snowflake SQL as a solution, linking to its documentation. A few commenters highlight that Excel can execute pivot tasks more efficiently than SQL, though SQL is not incapable. There's a reference to respecting the 'first normal form' in the database design. Also, users find the quality of life with SQL comparable to GROUP BY ALL function.

The Reddit thread discusses the potential of getting a job with just SQL and Python skills. Many commenters affirm that they built their careers on these skills, some even starting with just SQL. It's suggested that learning cloud platforms and ETLs in Python/SQL could be beneficial, and data engineering is highlighted as a growing field. The importance of adaptability, problem-solving, and a willingness to learn is stressed. Several useful books are recommended for further learning. Some commenters shared their experience of getting jobs through LinkedIn by highlighting their SQL skills, while others spoke about their salaries in tech roles. Practical exercises and networking are also encouraged. A common sentiment is that SQL and Python are valuable, transferrable skills in the industry.