Carmine's Kernel

Gopal Kakivaya: The Essence of Service Fabric

I recently got the opportunity to geek out with the founder and VP of Service Fabric. It was highly educational. It's amazing how much Gopal knows and it's equally impressive that he was not at all prepared for this interview - meaning he had no idea what we were going to talk about besides Service Fabric. Many executives have handlers who require questions in advance. Not Gopal. He's actually a VP who is also a hands-on engineer and architect. Of course, I filmed it (that was the point, really) :) Enjoy!

Gopal Kakivaya is the founder and Vice President of Microsoft Azure's Hyperscale Compute team, which is the home of Service Fabric, an open source, general purpose distributed systems infrastructure and microservices platform that powers several of Azure's core services. Of course, it's not just for internal use by the likes of Cosmos DB, SQL Azure, Intune, Skype for Business, Azure VM APIs, Event Grid, and more... It's tailored specifically for general purpose use for running cloud (micro)services of any size and complexity - authored by you.

The primary goal of Service Fabric, from a developer's perspective is, as Gopal says, "to make programming in the large (planetary scale) feel like programming in the small (your dev machine)". All of the distributed computing complexity of the cloud is abstracted away and you're left to focus on what really matters to you and your business: your (micro)service(s), your scenarios, your code.

So, how does Service Fabric solve some of the key hard problems of distributed computing, even abstracting them away completely from developers? Where did Service Fabric originate and where is it heading? What's the essence of Service Fabric?

Gopal gives an introductory tour of Service Fabric's subsystem architecture. We also discuss Service Fabric's built-in fault analysis services, the importance of verifying the behavior of cloud services under faulty conditions, and the future of Service Fabric's Chaos Engineering infrastructure.

Tune in. Enjoy. Learn.

 

More information:

Service Fabric Overview

Service Fabric Github Repo

Core Service Fabric Subsystem Explorer

Service Fabric Controlled Chaos

 

Service Fabric Chaos Engineering Service: We want to hear from you!

Greetings fellow chaos engineers and SREs!

I am currently working on adding capabilities to Service Fabric's Chaos engineering and experimentation service and I'm looking for feedback and insights from folks who are already using Service Fabric (but, of course, if you're not, please do check out our offering!).

Service Fabric is a general purpose distributed systems infrastructure and orchestrator (VMs, containers, etc...). It powers a lot of Microsoft Azure today including services like Cosmos DB, SQL Azure, etc...

Service Fabric was originally designed for internal (to Azure) workloads, but has been in the public domain for a few years now - and just recently we open-sourced it.

Service Fabric has a built-in (first class!) chaos experimentation subsystem as part of its Fault Analysis Service. SF Chaos is geared toward injecting faults that typically happen in clustered distributed system environments that are designed for extreme availability (node failure, quorum loss, replication failure, application level start/stops, etc...).

You can experiment with Service Fabric clusters in the cloud by using a free Party Clusters service if you want to deploy your SF application to a cloud environment without having an Azure account.

I'm looking to get more information on who's using our Chaos service today (we know many services are, but we don't know who they are) in order to get your feedback and insights, which will help in furthering the capabilities of the system. Also, if you have any questions, just ask. You can always get to me by emailing ctorre at microsoft... If you didn't know Service Fabric has chaos experimentation built-in and first class, now you do :slightly_smiling_face: Please let me know if you are using it and want to chat.

Cheers and happy chaos!

...Charles

Developing Manycore Applications with Concurrency Auto-Tuners

Continuing on with our coverage of ICSE 2011, meet Dr. Victor Pankratius. Dr. Pankratius heads the Multicore Software Engineering investigator group at the Karlsruhe Institute of Technology, Germany. He also serves as the elected chairman of the Software Engineering for parallel Systems (SEPARS) international working group. Dr. Pankratius' current research concentrates on how to make parallel programming easier. His work on multicore software engineering covers a range of research topics including empirical studies, auto-tuning, language design, and debugging.

In this video, Wolfram Schulte joins Victor to discuss the challenges of making concurrency easier for developers. One of the really interesting approaches that Victor and team are investigating is concurrency auto-tuning, and the example discussed here involve adding OS kernel-level support for auto-tuning user mode applications for manycore processor architectures. This is very fascinating research with great potential. Concurrency auto-tuner in an OS kernel? Concurrency-enlightened operating systems? Why not? Always great to meet young innovators with no fear of failure. I hope to see this type of thing materialize. Very interesting research and real world problem. Go Victor. Go!

Thanks to Wolfram and Victor for another great conversation. Tune in.

C++ Renaissance

When you think about all the code executing in the world at any given time, there's a good chance you're thinking about a lot of code written in C/C++ (aka native code). As Mohsen Agsen mentions so astutely in this impromptu and candid conversation, C++ can be thought of as the dark matter of the developer universe: there is so much of it all around us, hiding in the computational shadows, powering so much of what we take for granted, technologically.

C++ is currently undergoing a renaissance. This means that, by definition, the language, compilers and compositional tooling are evolving and coalescing into a state that maximizes native developer efficiency, productivity, and creativity across hardware and software domains (PCs, mobile devices, embedded systems, operating systems, user applications, services, etc). C++ is a powerful "systems" programming language, but it's more than that. It's object oriented, but it's more than that. At Microsoft, most of our flagship products are written in C++ (and C, like the Windows kernel...). As somebody with a keen interest in programming languages and software engineering, generally, I wanted to get some answers to broad questions concerning the language that consistently ranks near the top of the most widely used general purpose programming languages in the world. Who better to talk to than some key technical leaders driving Microsoft's Visual C++ business?

Mohsen Agsen is a Microsoft Technical Fellow and veteran C++ developer who runs the VC++ engineering team. Craig Symonds is the Director of Program Management for VC++ and a long time Microsoft dev tools veteran. Both Mohsen and Craig have been at the company for many years and have a ton of industry experience. I paid them a visit to see what’s on their minds these days regarding the native developer community, C++, Visual Studio, and more. As you will learn, Microsoft and Visual Studio, specifically, are re-doubling efforts to take part in the native code renaissance. Accordingly, you may see advances in our native tooling that the team thinks of as “C++ first” -> VC++ will extend its capabilities on a faster pace than it has ever done so in the past, at times surpassing the other VS languages/runtimes, in specific scenarios. This is exciting and a long time coming, but of course it's more passion than promise at this point. There is no specific news here, just perspectives and insights among some very bright people driving Microsoft's C++ efforts. I really enjoyed the conversation with Craig and Mohsen and hope to chat with them again in the near future.

Links into specific conversation points (video skimming):

Introductions
State of the union for C++ from Microsoft's perspective
C++ as both a low level and high level programming language
C++ as object oriented, but OO is a capability, not a requirement
Garbage collection in C++. Why not?
C++ portability
Visual Studio C++ integration and parity with the other languages/runtimes in VS and C++ first

Here’s to the C++ Renaissance.

Tune in. Enjoy.

C

100th Episode of Going Deep!

Welcome to the 100th episode of Going Deep! It's been a great ride so far and I hope you've been enjoying the show over the years. For this momentous occasion, meet Bart de Smet, a software engineer extraordinaire on the WPF team who spends his free time blogging (what an incredible wealth of truly useful technical information to be found on Bart's blog!) and creating custom LINQ providers. In fact, Bart is probably the world's most prolific LINQ provider creator, from LINQ-to-MSI to LINQ-to-Simpsons! How does he do it???

Of course, in this special episode who better to have involved in this LINQ'ified conversation (with lots of whiteboarding) than LINQ co-creator, programming languages designer, fundamentalist functional programming high priest and Channel 9 star Erik Meijer? In fact, given that this is an Expert to Expert Going Deep, Erik co-conducts the interview (and he's great at it as usual!). Nothing better than watching and listening to two experts geeking out at a whiteboard!

What makes LINQ so readily "providable"? How does Bart concoct his LINQ provider magic? Here, we dig into the details that enable LINQ-to-Anything.

Tune in. There's a lot of very useful technical information here (which I hope is the case for all 100 Going Deep episodes).

Enjoy and thanks for watching!

Software Composability and the Future of Programming Languages

How will imperative programming langauges evolve to suit the needs of developers in the age of Concurrency and Composability? What role can programming languages play in enabling true composability? What are the implications of LINQ on the furture of managed (CLS-based) and unmanaged(C++) languages? How will our imperative languages (static) become more functional (dynamic) in nature while preserving their static "experience" for developers?

Answers to these questions and much more are to be found in this Channel 9 interview with some of Microsoft's leading language designers and programming thought leaders: Anders Hejlsberg, Technical Fellow and Chief Architect of C#, Herb Sutter, Architect in the C++ language design group, Erik Meijer, Architect in both VB.Net and C# language design and programming language guru, and Brian Beckman, physicist and programming language architect working on VB.Net.

This is a great conversation with some of the industry's most influential programming language designers. Tune in. You may be surprised by what you learn...

Programming in the Age of Concurrency: The Accelerator Project

David Tarditi and Sidd Puri, scientists at Microsoft Research in Redmond, WA, USA, are creating some really compelling managed libraries that provide an elegant approach to writing highly parallel data intensive code. They basically hack the GPU to enable roaringly fast parallel computation. They call this .NET-based framework "Accelerator" and explain it in detail on Channel 9. Check it out and get the bits in your hands now. This is great stuff.

Concurrency and Coordination Runtime (CCR)

We just released a great video interview on Channel 9 that should be of tremendous interest to those of you who write managed threaded code: The Concurrency and Coordination Runtime (CCR) is a lightweight port-based concurrency library for C# 2.0 developed by George Chrysanthakopoulos in the Advanced Strategies group at Microsoft. Here, we have a deep discussion about CCR with George, a Software Architect, and Satnam Singh, Architect. You can get more info about CCR on the CCR Wiki. This is super cool stuff and represents a really innovative approach to making managed threaded programming more readily understandable and predictable.

Please check out the OOPSLA/SCOOL paper on the CCR.

Enjoy!

C

Spot the Bug on Channel 9

If you're a developer, check out this Spot the Bug post on Channel 9. These will appear every week or so (maybe longer intervals, depends on juiciness of the code sample). Enjoy and spot those bugs (there's a few of them...)!