CNCJ - April, 2021

How to think Cloud Native for your next Data project?

Summary

We invited cloud-native specialist Harry Lee and data specialist Archana Arakkal to chat about how the two ecosystems are growing to serve the community. And how you, as a practitioner, can start thinking cloud-native to improve your efficiency!

Here’s is the timestamped breakdown of the items discussed:

  • 5:10: What are some examples of data projects in public, hybrid and multi-cloud environments?
  • 8:52: What is the difference between cloud, cloud-computing and cloud-native?
  • 12:45: Examples of how cloud has accelerated data science projects?
  • 17:15: If cloud is so awesome, how come not all data projects are on the cloud?
  • 19:20: What can we learn from the software world to accelerate cloud adoption for data projects?
  • 21:15: Anecdotes of what you can give to the data science community who really want to move into cloud but are stumbled by the various reasons
  • 24:07: Do you see people using cloud-native technologies in data science projects?
  • 26:15 What is the cool trend and technology that you see the data science community is adopting in the industry?
  • 30:03 How to adopt a typical cloud-native architecture (such as microservice, declarative APIs, etc) to data science project?
  • 32:37: How would you transition a traditional ML project to a cloud-native ML project?
  • 36:10: When does it not make sense to adopt cloud-native approach for data science project?
  • 40:50: What is your experience with re-architecting data science projects, and what is your anecdotal advice? (Archie)
  • 44:40: What is your experience with re-architecting data science projects? What to do and not to do? (Harry)

Questions from audiences:

  • 46:45: How do you choose between the various technology betwween vendor-specific, open-source, hybrid? How do you choose these technologies based on cost and effort?
  • 52:48: With Kubernetes bleeding into every technology spectrum, what does it mean for data professionals? Do we all have to learn DevOps or do we have to all hire DevOps engineers?
  • 54:40: Do you feel that having a software engineering background helps your professional journey as a PhD candidate in specialising Federated Learning?
  • 59:20: From a developer’s perspective, should everyone also learn data science?
  • 1:04:00: What is your view on having more and more citizen data scientist?
  • 1:08:21: Once you have a product that is fully running in production, how do you guarantee that the product is running as expected from a monitoring perspective? What would be your opinion on an SRE in the space of data monitoring?
  • 1:14:50: Can we integrate data science monitoring to existing software monitoring tools that you have available?

Session Recording