How to think Cloud Native for your next Data project?
Summary
We invited cloud-native specialist Harry Lee and data specialist Archana
Arakkal to chat about how the two ecosystems are growing to serve the
community. And how you, as a practitioner, can start thinking cloud-native to
improve your efficiency!
Here’s is the timestamped breakdown of the items discussed:
5:10: What are some examples of data projects in public, hybrid and multi-cloud environments?
8:52: What is the difference between cloud, cloud-computing and cloud-native?
12:45: Examples of how cloud has accelerated data science projects?
17:15: If cloud is so awesome, how come not all data projects are on the cloud?
19:20: What can we learn from the software world to accelerate cloud adoption for data projects?
21:15: Anecdotes of what you can give to the data science community who really want to move into cloud but are stumbled by the various reasons
24:07: Do you see people using cloud-native technologies in data science projects?
26:15 What is the cool trend and technology that you see the data science community is adopting in the industry?
30:03 How to adopt a typical cloud-native architecture (such as microservice, declarative APIs, etc) to data science project?
32:37: How would you transition a traditional ML project to a cloud-native ML project?
36:10: When does it not make sense to adopt cloud-native approach for data science project?
40:50: What is your experience with re-architecting data science projects, and what is your anecdotal advice? (Archie)
44:40: What is your experience with re-architecting data science projects? What to do and not to do? (Harry)
Questions from audiences:
46:45: How do you choose between the various technology betwween vendor-specific, open-source, hybrid? How do you choose these technologies based on cost and effort?
52:48: With Kubernetes bleeding into every technology spectrum, what does it mean for data professionals? Do we all have to learn DevOps or do we have to all hire DevOps engineers?
54:40: Do you feel that having a software engineering background helps your professional journey as a PhD candidate in specialising Federated Learning?
59:20: From a developer’s perspective, should everyone also learn data science?
1:04:00: What is your view on having more and more citizen data scientist?
1:08:21: Once you have a product that is fully running in production, how do you guarantee that the product is running as expected from a monitoring perspective? What would be your opinion on an SRE in the space of data monitoring?
1:14:50: Can we integrate data science monitoring to existing software monitoring tools that you have available?