Property Based Testing with Jqwik
Video
Code
Cheers,
Diego Pacheco
Showing posts with label FP. Show all posts
Showing posts with label FP. Show all posts
Friday, July 24, 2020
Wednesday, July 4, 2018
Having fun with Cats
The Cats Structure
Cats always have this Structure:
- A Type Class: Which is a trait in Scala
- A Type Instance: Which is an Object in Scala
- A Type Interface: Which is an Object in Scala
- An Implicit Class: Which use an implicit class in Scala for better syntax
This way there is complete decoupling from the Alberic Type(Type Class) from Type Instance which is the binding for the Types you want to support and Type Interface which is the functions that will be generic for all your supported types. It's possible to add more Types to this system without having to touch previous classes that why type classes are much better for extension than OOP. Let's take a look into a code sample to get this pattern right.
Here we have all patterns together in the same code sample. The type class here is called Show, the idea of the show is to provide contract on how to Show some type, which might be printing to the console or logging into a file or send to a TCP port for the centralized log, it does not mapper function is decoupled from Type here. Them we have our Type Instances, there are 2 type instances: for Int and for String, this means we cant show Double for instance, If you want show Double you can provide the Type Implementation and that happens outside of this file so as you can see this is super extensive, correct and backed by the compiler. Them we can the Type interface which is the ShowInterface that for this case is how we will provide functionality for the Type. Finally, we have an implicit class in order to provide better syntactical sugar.
A Bit of Theory and Definitions
Cats have Type Classes like SemiGroup, Functor, Monoid, Applicative, and many others and also has Data Types like Either, Eval, Ior, Id, Kleisli, State, and many many others. Data Types inherit from each other and this will the pileup functions. Pretty much all start with a SemiGroup which is just a Type that implements combine and it's capable to combine 2 Types of the Same Type. So Monoid is just a SemiGroup that provides the empty function. Cats provide type instances for most common and useful type in Scala such as Lists, Maps, Option, Either, Try, Future and many many others. So you don't need to define the algebra and functions for these types since Cats provide that for you out of the box, however, you might need provide evidence(implementation) for your Own business types. Functor is something that provides map implementation and is great to apply effects to data. There are type classes instances for Functors, Monoids, and Monads from the same common types I mentioned before. Type Classes have laws which need to be true always, Laws are something that gives your guarantee of composability however they don't help you to understand what the type is or what the type is doing. Some types in Cats are very straightforward and easy to understand like SemiGroup and other are very complicated like Cons.
Scala and Cats
Let's see how easy(for this samples :-)) is for you to use Cats with Scala.
Monoid Sample - Here we are using Option and monoid and combining 2 options.
Either Sample - Here we are using Int as Either with nice syntax and combining them with For Compreenshions in Scala.
Functor Sample - Here we have Functor Lifting where we have a normal function and turn into a Functor without having to implement type algebra.
That's it - I hope you had fun and that trigger your curiosity to Learn more about Cats and Function Programming Patterns, Math and Decoupling. I'm not an FP Zealot, however, I see the benefit in Theory so I search found the balance between Theory and practical programming. If you might wondering, does anyone use this kind of ideas in production at Scala? Yes, Twitter, Yelp, Verizon and many other big companies, These ideas used in Spark and Big Data for instance as well.
Cheers,
Diego Pacheco
Wednesday, June 27, 2018
Scala Monads 101
Why Monads are Hard?
IMHO it's because is all about Math and the fact that historically Functional Programming started at the Academia and there is much elder FP material on the internet rather than now when I say now, I mean that now we live in a post-functional era where things are more practical. Monads are very similar to Functors since both are Wrappers. Wrapper to me is the best definition of a monad/functor. Imagine you have some data and you want to wrap that data around with some Context so you can provide extra meaning or extra functionality, in order words understanding and/or functions.
Going back to the math, sorry, just a little bit more... Functor means that you apply a function(called map) to a wrapped value. What's the difference to a Monad? Well, monad you use flatMap :D So It's possible to map a function with another function, this means 2 things, first of all, that we are doing function composition and second that functions are functors too. So we can end up with functions and values wrapped by context. In order to get a value wrapped in context and apply into a function wrapped in Context, we need an Applicative which has the function apply. Monads apply a function(which return wrapped value - also know as flatMap) to a wrapped Value. So monads are about flatMap. So Applicative and monads are the same things? Similar but Monads are more powerful since you can get the result from one computation and pass to another one.
Let's t take a look at how to define a monad type in Scala.
Here we have 2 functions: pure and flatMap. As you can see monad takes type parameters so we can specify proper types when we do concrete implementations. Concrete implementations are very simple and easy to do. The function pure also know as apply or unit means that I will pass the raw value of A and A will be wrapped by a Monad called Monad[A]. The other function flatMap does the same but the difference is that the value comes from a function that wraps that value rather than the pure value. That's why is called flatMap because the function knows how to transform/map the computation to the wrapped value.
I won't cover monadic laws here in order to not make this more complicated than already is but if you curious about it there is the great post about Monadic Laws in Scala. Besides implementing pure and flatMap a monad has 3 properties that need to be always true, that's are the monadic laws.
Let's see a Theorycal application of monads before we see a practical one, let's say when to want to wrap numbers with classifications like Even or Odd. We can write simple monad to do that, let's take a look on the code.
Here we have a Monad trait with 3 monads implementations, where we have a companion object that creates the proper monad based on the fact that the number % 2 ==0 is even otherwise is odd.
Practical Usage in Real Life
Future and Option are great samples of real-life usage of monads. You can create your own monads as you find need to provide proper abstractions and functionally. Scala has ScalaZ which is a library who come full fledge with Functional Programming and Category Theory principles. List and Set are monads too in Scala. It's very likely at this point your convinced that monads are real and you already used them a lot.
Let's take a look on a practical real-world example.
Here is this sample you can see we have 3 services, we have a service to book plane tickets, we have a service to rent a car and we have a service to book a hotel. We are doing a big function composition using flatMap so the computation of one stream goes to the other stream. In order to have the code in a more clear code, we use for comprehensions in Scala. This is nice because this code is verified by the compiler and is all async but is async in a safe way which means verified by the compiler.
I hope this helps you out to digest monads and find the middle ground between mathematical theory and practical system development. Needless to say that these concepts are getting bigger than Scala and Haskell since pretty much every single language today is having some Functional influence now.
PS: There is an interesting fact about the image I used for the cover of this blog post, that image is the graphical representation of a monad following category theory.
Cheers,
Diego Pacheco
Wednesday, May 9, 2018
Code for Reliability
What is Reliability? What do I need to change?
Wikipedia definition of Reliability.
As you can see reliability is described as "trust", like something you can trust. So what trust would be in the context of distributed systems and cloud-native? Well IMHO means that you can resist several kinds of failure. It's easy to imagine failures on the macro level(External) such as:
- A Data Center(Region) fail
- A Microservice Instance Fail
- A machine Fail
- Timeout Happens
- Latency increase
- A service hangs and never returns
- A service breaks the data protocol and returns dirty results
- A downstream component fail and that creates issues for your service
However, Reliability also needs to be applied inside the system, micro level in other scenarios such as:
- Lack of internal validations(i,e: expect an email and receives a number)
- A database call fails: You try to persist but the client/server fails
- You get wrong input data and your code blows(parsing issues - i.e: json/yaml)
- A corner case in your code make you fall (untested corner case)
- You were expecting one exception and another one happens(bad error handling)
- Some HTTP call timeout and you were not counting on that(i.e lack of retry)
This means we can apply "degradation" thinking on internal design as well. This is a mindset and when we are coding there are decisions we made that can make your code more or less reliable.
Being Explicit
Being explicit it's a very functional way of thinking. So basically you should avoid having methods that return void. Methods returning void are hard to test and often have side-effects. Being explicit it's not only about what you return but also what expectations you create on your caller. Interfaces are ways to describe behavior and simple mechanisms to inform right contracts. Every single class has a default interface which is the public methods, even if you don't attach new interfaces you already have a contract. External interfaces are interesting because they could provide additional meaning to the call, for instance, you can implement: Retryable or Cacheable. When you do that you inform your consumer that piece of code will be trying more than 1x and you will cache results. Documentation is one dimension of explicitness however if you can translate the documentation in metadata like interface and annotations(talking about java) you will end up being more explicit.
The pseudocode code above express explicit some behavior that the class will apply. Interfaces are not only for sake of contract but also for sake of "demarcation" and behavior enforcement.
Safe Return
Basically, that are 4 options when we think about safe return. The first thing you can do is never return null. Null is dangerous because you can get an NPE(Null Pointer Exception) also null is bad because you cant distinguish null from error. So the second thing on being explicitly about return is throw exceptions(Runtime based) when there are wrong state or bad input parameters. The third thing is, going back to the null thing, you can return good defaults in case you can do the proper processing. The last but not least is that you can return Optional. The option is a monad(FP concept) which means that you might be the right or wrong result. The great thing about the Optional Monad is that make that possibility explicit for the caller.
Here are being explicit with the fact that you can call this code in a multi-thread scenario without issue because we have no shared-mutable state, therefore, this code scales and is ThreadSafe as the interface explicit tell us.
Retry and Timeouts
Every single HTTP call you make or network call(no matter the protocol) should have Retrys, timeouts and sometimes exponential backoffs. This is a standard SRE practice. Check this great material from google GCP. If you are dealing with OS process or something that might fail due networking is already a good idea to have this mechanism in places.
Validations and Limits
Everything needs to be validated everywhere. So often we have a multi-layer architecture like you have a front end, backend, multiple services, adding validations in all ends at least for simple input parameters when possible save latency and network round-trips but also improve the user experience taught a faster response. Validations make your code more resilient however we need also define limits, every time you can UNBOUNDED-* you will run into problems. In order to fix this problem, we need to define limits.
Everything that is not caped(has limits) will eventually overflow. The overflow of one system might create an overflow in other systems - that's why we need to have CONTENTION and once the failure happens we to stop the failure instead of cascading failures to other components and service. We will cover this in more detail in next section.
Try/Catch as Resiliency Operator
Try/Catch should be as granular as possible. A big try-catch block can be dangerous. In the other way around a big try/catch block make sure your code doesn't blow. So you need to think about try/catch as the basic mechanism for RETENTION and ISOLATION. Fine-grained try/catch allow you to know precisely where your code is not working thus you can have a better understanding, logging, and observability if you publish internal metrics accordingly.
As you can see here we are calling multiple service/component each service might fail. But if I can get a CarRental recommendation that should not make the booking experience to fail(which would be very bad for the user and user UX). So having fine-grained try/catch allow us to catch whats going one. This is a very naive sample, in real life, you might do things in async(non-blocking IO way) nevertheless you would still have try/catch expressed in the form on callbacks and still don't blow the whole process if one part fails. This leads us to NetflixOSS Hystrix which we will talk more about on Fallbacks section.
Good Defaults and Self-Tuning Systems
Good defaults often save time(configuration time) and also require less parametrization from the caller. Having the ability to change defaults at Runtime is a must have. The higher level of system maturity is when you can have the system perform self-tuning and change the internal defaults(thresholds) ad the system gather data from the users and assume better defaults. This could avoid downtimes, redeploys and extensive tweak and reset cycles. In order to understand if the system is self-regulating well - great observability is required.
Most of the times you won't need to code the WHOLE self-tunable system like ASG. However, you need to TRACK and PUBLISH internal system metrics to Cloud Watch for instance so the ASG can auto-tune your component. Good defaults are more basic but also important in the sense that they can reduce configuration complexity.
Fallbacks
Fallbacks are the holy grail of internal system reliability. That's when we need total about Hystrix. Fallbacks are like catch blocks but with better instrumentation, since Hystrix already provide Thread, Cache, and Metrics for each command. This is much better and much more sophisticated mechanism then try/catch as we discussed before. So whats important here(when possible) is to have multiple levels of fallbacks. This is the ultimate resiliency property. So if you thing fails you try, another and another and another and another(multiple fallbacks before giving up).
It's important to keep in mind that fallbacks should be simple. However, in order to provide more resiliency, we need more code. Fallback can fail and you need to address that. I hope all the concetps and ideas halp to shape your coding midsets and make you build more resilient systems no matter if you are using netflixoss stack or not.
cheers,
Diego Pacheco
Sunday, November 19, 2017
The power of Scala Type Classes
There are always several ways and styles of thinking to address problems. Type classes is another way of thinking and dealing with problems. For some problems, you might just use Pattern Matcher -- But if you have to add a type? Right, I could stick with good old OO since Scala is hybrid right? -- But what if you need to add one operator? Type Classes can help you out.
What Are Type Classes?
In order to have a type class you need to have:
1. A signature
2. All implementations for Supported Types(You can add more types later)
3. A function that requires a Type Classe
What are the Use Cases for Type Classes?
Type classes are great because of the make behavior more explicit and extendable. This "Behavior" more explicit happens because this will be checked at Compile Time. So you can extend and add more Behavior without re-compiling or be changing your previous code. That's why type classes are powerful.
What are the practical Use cases?
1. Business Rules / Domain Objects: Where you need to express different and customizable calculations. Doing this changes without breaking the client.
2. Pipelining: When you need to apply several different rules or a request.
3. Serialization/Deserialization: When you need express different parsing logic.
Sample Code
Now let's see some sample code. First code we will define some simple Math operations and we will use Implicits to apply automatic conversion for use.
So here we have a Trait called NumberLike witch is expecting a Class called T. Which could be any numeric type. Them we can a companion object with added support for 2 types Int and Double. This support is added via implicit objects called NumberLikeInt and NumberLikeDouble.
Finally, on the same object, we have 2 implicit conversions one to convert an Int into a NumberLikeInt and another to convert Doubles in NumberLikeDouble. Once we did the import on the main method we can use regular Int and Double add call our talk math methods.
The second sample is about behavior with different objects and the ability to add things later. So let's take a look at the following code.
Here we have 2 case classes: Person and Cat. We also have a Trait Can talk which expecting a Type T where T can talk. On AddOns companion object we have some default behavior implementations like PersonTalker and CatTalker witch allow Person and Cat to talk.
We also have an implicit class wrapper which gets a Type A and parameter X. This class provides the talk method where the CanTalk[A] will be implicit and will delegate to method talk passing the X argument.
Right after doing the write import we can call talk method on case class Person and Cat. We also can add new types so I defined a Dog after the fact.
Scala Type classes are really powerful and cool it allows to do really modular design and extend the design in a very clean and concise way. You can download the full source code and SBT project on my github here.
Cheers,
Diego Pacheco
Thursday, August 10, 2017
Functional and Reactive Programing with Java 8
Java 8 got some sort of FP influence, Thanks to Scala I would say. I suppose to deliver a talk about functional and reactive programming with Java 8, 2 years ago and I found this slide deck in my machine so I thought about making a video presenting some of the functional programming principles with Java 8 including frameworks such as RxJava. So this is a ~40 minutes video - I hope you guys enjoy and have fun.
Video
Functional and Reactive Programing with Java 8 from Diego Pacheco on Vimeo.
Slides
Cheers,
Diego Pacheco
Thursday, May 7, 2015
Core FP Concepts slidecast
Functional programing is growing a lot now a days, not only pure function programing languages but several new languages are hybrid or at lest had fp influences like java, javascript, c#, python. On this slidecast i will talk about some of the core concepts on functional programing, here you want find all concepts but this could be a starting point for you.
I hope you have fun :-)
Video: https://vimeo.com/127154380
Slides: http://pt.slideshare.net/diego.pacheco/conceitos-funcionais
