Scala Currying & Partially Applied Functions

Currying is a concept rooted in mathematics and has also found it is way in computer science. Currying is a process of translating a function taking multiple arguments to a set of functions accepting only one argument. It is named after Haskell Curry! Currying in computer science is also done in a similar manner. You can read more on the background here. h(x)(y)(z) = f(x,y,z) h = curry(f) Let’s see how we can apply currying in Scala. Well, it’s not difficult to curry a function. See Below If you just pass addCurried(10) you would get an error. Which is not good and to be fair the function did require all the parameters to evaluate. Getting around this is simple with partially applied functions, these should not be confused with partial functions. See Below Another example of a partially applied function How about currying an existing function. That is also possible via Function.curried. It … Read more

Scala Partial Functions

As I was learning Scala and first read the phrase Partial Function had a weird thing about it. How can a function be partial??  I started digging into the documentation and online. What came out was a surprisingly very clean solution to a problem which every programmer encountered. Every function or method which is written usually applies some validation rule to make sure that the parameters which are passed to it are valid. It makes the code a bit messy. We had the logic validating the parameters and then the actual business logic which is invoked if the parameters were correct. That means that the function only works on a subset of parameters. Partial functions provide a much better solution. So now let’s look at the definition of a partial function. Quoting from Scala Doc A partial function of type PartialFunction[A, B] is a unary function where the domain does not … Read more

Scala Functions – Higher Order Functions & Function Literals

Higher Order Functions are supported by many languages including Haskell, Scala, F#, Go, Javascript. Higher Order Function is a fancy name for a function which accepts functions as parameters.!! How is that? Well – that is because functions in Scala are treated as first class values – that means in Scala they can be assigned as values, passed as parameters and returned from functions. Functions are said to be higher order functions if they do atleast one of the following Accept functions as parameters Returns functions as a result The map function is one of the most popular higher order function.  Most Scala programmers have come across this function in their lives. Before we get any deeper into functions – let’s look at another type of functions called function literals. They are more famously known as anonymous functions. Function literals are functions which do not have any name and are created, passed and returned … Read more

Scala Functions – Introduction

Introduction Functions in any language can be defined as a group of statements that constitute a logic which is performed repeatedly during the execution of a program. Functions in scala are no different.Scala identifies both functions and methods. They are both treated a bit differently. Finally, scala does support functional programming(FP) paradigms and this blog entry tries to take you through some of the FP paradigms and get you a good brief. They follow the following format def functionName ([list of parameters]) : [return type] = {  function body  return expression} An example of  a function in Scala. See Below In addition to that, scala also simplifies creation of code. In functions,  it removes the need to write boilerplate code. Every programmer likes less code. Speaking of less code some of the stuff scala does away with  is No need of semi-colons Simple functions do not need curly braces. return … Read more

Scala Collection – Operators

To be honest there are no operators in Scala :). Every method in scala is an operator! But for some simplicity let’s just call them collection operators. Below are some of the operators which are common to all scala collections. The examples are mostly done using Arrays but can be applied to collections as well. length operator distinct – More importantly it returns an array of elements. range, fill head, tail, last head operator returns the first element of the array. tail returns all the elements but the first element.last returns the last element of an array Let’s look at some more interesting ones filter is a transformer and returns a new collection. It requires a predicate function which is applied to every element in the collection and if it evaluates to true then that element is returned. filterNot is exactly the opposite and would return if the condition evaluates to false. In … Read more

Scala Collections – Streams, Maps & Sets

Streams Stream is similar to a List however, it can have unlimited elements. It is a Traversable but lazy in nature. A Stream in Scala consists of a head and tail which is lazily computed i.e elements are only evaluated when they are needed. Hence it can be inferred, laziness helps to make streams infinite. Transformer functions like map, filter etc are also applied lazily although care should be taken that not all functions are lazy – i.e max, sum etc. Elements can be added to a stream as below. Keep in mind that when you are adding an element to the stream we also need to append Stream.empty Additionally also checkout the #::: that is only required after the stream name and then it is #:: Transformer methods are lazily evaluated as show below – e.g filter Transformer methods are lazily evaluated as show below – e.g map Sets Set … Read more

Scala Collections – Lists, Sequences & Vectors

Below, I have provided some of the collections which are usually the most used. Starting from the regular ones like Sequences, Lists, Maps, Sets we go on to discuss some of the more interesting ones like Vectors, Streams. Lists List is probably the easiest to understand and hence most used data structure/collection. In scala a list is implemented as a class. I quote A class for immutable linked lists representing ordered collections of elements of type A By default, all lists are immutable. However the devil is in the detail. Check out the scala documentation here. Despite being an immutable collection, the implementation uses mutable state internally during construction. These state changes are invisible in single-threaded code but can lead to race conditions in some multi-threaded scenarios You can declare lists in various ways and they are documented below Lists are efficient when the programs which use them are only … Read more

Scala Collections – Introduction

Introduction I started looking at Scala collections after realising a need to polish some rough edges in the quest to become a better scala programmer. The scala collections series of blogs are my aim to become better at scala. It has been written from a beginner’s perspective but I hope I would be equally helpful to seasoned programmers. Scala collections are a vast a topic and there is something for everyone if you know where to start. There are few commonly used collections that are described here and is a good place to start. There are many collections which are not covered and hopefully this blog will give you an idea as to where go and look and how to implement them. Let’ start – Scala collections have a very well defined & documented hierarchy available here. However, I have taken the liberty to insert a few images from the … Read more

Apache Airflow – First DAG

Now that we have a working Airflow it is time to look at DAGs in detail. In the previous post, we saw how to execute DAGs from the UI. In this post, we will talk more about DAGs.DAGs are the core concept of airflow. Simple DAG Simply speaking a DAG is a python script. Here is the code of a hello world DAG. It executes a command to print “helllooooo world”. It may not do much but it provides a lot of information about how to write an airflow DAG. Understanding an Airflow DAG Remember this code is stored in the $DAGS_FOLDER. Please refer to the previous blog which has the details on the location. An important thing to note and I quote from the airflow website One thing to wrap your head around (it may not be very intuitive for everyone at first) is that this Airflow Python script … Read more

Apache Airflow – Getting Started

I recently finished a project where Apache Airflow(just airflow for short) was being touted as the next generating Workflow Mangement System and the whole place was just going gaga over. Well, that got me thinking how I could get to understand and learn it. Hence the blog post. Here are some things you may want to know before getting your hands dirty into Apache Airflow What is Airflow?The definition of Apache Airflow goes like this Airflow is a platform to programmatically author, schedule and monitor workflows. Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. So Airflow executes a series of unrelated tasks which when executed together accomplish a business outcome. For those folks who are working on the likes of Informatica – airflow is similar to Workflow Designer or those working in Oracle Data … Read more