Nagaraj Tantri

This blog is a quick code snippet, which I thought of sharing!  So the below method/function is to ensure to convert a device id to MD5 version and before doing so, convert the data column to upper case and convert the value to hex and again upper case. Google's Ads Data Hub expects you to convert the data to the below format: UPPER(TO_HEX(MD5(UPPER(device_id)))) as device_id_md5 The below code helps us to achieve that in PySpark: from pyspark.sql.functions import md5, col, upper, hex from pyspark.context import SparkContext from pyspark.sql.session import SparkSession sc = SparkContext('local') spark = SparkSession(sc) df = spark.createDataFrame( [ ["63c94c81-44e4-4e4d-bb26-b95648581a16"], ["c5628aa9-92de-4e7d-ac60-e35851e93f22"], ["eeb472cb-aa3a-44f1-818c-bcc069d57367"] ], ("col1") ) df.withColumn("col1", upper(hex(md5(upper(col('col1')))))).show() NOTE: The above device ids are dummy ones.

How to change bindIp for Mongo in MacOS Mojave A lot of times when you figure out that the local Mongo server you wanted to connect to was not getting connected from outside of your laptop or from the docker based application, it means you have to change the local bind IP address. To ensure that you can connect to the local servers, you have first to change the config file: /usr/local/etc/mongod.conf  systemLog:    destination: file    path: /usr/local/var/log/mongodb/mongo.log    logAppend: true  storage:    dbPath: /usr/local/var/mongodb  net:    bindIp: 0.0.0.0 Change the bindIp from 127.0.0.1 to 0.0.0.0 Stop the service using:  brew services stop mongodb-community Start the service using:  brew services start mongodb-community

At our company, we have many requirements for enabling cross-account access with AWS. This requirement was around for usage of S3 and accessing the data from S3 for our in house Ingestion Service. To give a brief about Ingestion Service, it’s part of our Datamanagement ecosystem which is used to ingest data from different source platforms and dump the data to S3. These different source platform also includes AWS S3. Our existing service on our AWS EC2, when it requires access to S3 files, stored in a different AWS account, we find it difficult to keep updating our access/secret keys. As a couple of companies have a requirement to rotate IAM user credentials over a defined period. We have utilized the cross-account IAM role access strategy, wherein the roles can now assume role via temporary credentials (obtained every time we query). How to set it up? Create Role : Set a role from your current account, which would be shared to the third party clients. Allow that r

Load test using Apache Bench We have been using Apache Bench to load test our rest microservices. One of the most useful situaiton while performing a load testing, is to change the number of concurrent requests parameter. Apache Bench is a simple tool which has helped us to do this quick testing to see whether our services break on huge load. How to Install Apache Bench on Ubuntu - 18.04 LTS In order to install Apache Bench all you have to do is: $ sudo apt install apache2-utils How to execute "ab" for POST API First create the JSON body to be posted, add that json data to a file with follwoing syntax: $ cat my_data.json json='{"queryType" : "groupBy","dataSource" : "pixel-feed","dimensions" : [], "granularity" : "all","filter" : {"type" : "selector","dimension" : "pixel_id","value" : "122112"},"

Introduction Often we keep getting confused as to why use one over another. What are the actual differences!  So I have tried to consolidate some of the basic differences in this blog. Type Docker (Container) Virtual Machine Behavior of application Run Isolated Isolated Libraries Needed (example: curl, glibc) Uses it's own Uses it's own Kernel Host's kernel Own kernel Operating System (OS) Host's OS It's own OS acting as Guest OS Docker kernel is same as Host's kernel? Since all Linux distribution's use same underlying Linux Kernel and only differ in their user view (view can be UI or application software's). It is pretty easy to add different user view over another Linux OS kernel. That's the main reason an OS on top on another OS is usually the case for virtual machine. Due which, we c

React ( Docs ) React makes it painless to create interactive UIs. Design simple views for each state in your application. React is Declarative: Imperative would say (like JQuery): if (user.likes()) { if(!hasBlue()) { removeGrey(); addBlue(); } } else{ if(hasBlue()) { removeBlue(); addGrey(); } } Declarative would say: if (this.state.liked) { return <bluelike/>; } else{ return <greylike/>; } We now can use ES6 and a summary of features is  http://es6-features.org/ Component Life Cycles: Mounting These methods are called when an instance of a component is being created and inserted into the DOM: constructor() componentWillMount() render() componentDidMount() Updating An update can be caused by changes to props or state. These methods are called when a component is being re-rendered: componentWillReceiveProps() shouldComponentUpdate() componentWillUpdate() render() componentDidUpdate() Unmounting This metho

Introduction There are many test frameworks for testing React app.  Facebook introduced their own Jest framework which helps you run test cases against the React components and also mock some of the flow.  With lot of JavaScript developers in the community also having their due soft corner towards Mocha as their test runner. Lets see the things that are good and can be the right tool for the environment.  This post is basically trying to give more overview about how we can test a react app and ensure to automate the same. Tools for testing components The most favored tools include the Jest and Mocha. There are also plenty of other tools like Jasmine. Now these are all your test runner frameworks which use plenty of other libraries to get their work done, like Chai, Karma, PhantomJS or JSDom. Checking this webpage tells more about the community ​ usage : Talking briefly about Jest and Mocha (which requires other library support to run the test cases)

It's something I keep using a lot! The Chrome's Secure Shell app is wonderful and is handy for people who want to use the chrome's tab for connecting to the remote server. So here it is: Install  Secure Shell  from the chrome web app store. Once added, open the app and select "[New Connection]": Enter the details: Remember to change the permission of the pem file, using the following command:  chmod 400 mykey.pem Generate the public key from the private key:  ssh-keygen -y -f mykey.pem > mykey.pub RENAME the private key (.pem) file to remove the extension of the private key ( The most Important :P ). Otherwise, the files won't be accepted by the plugin:  mv mykey.pem mykey Click on the "Import" button beside the Identity in that screen and select both the files: In case you want to Reset the known_hosts  then do the following

Recently I was working on creating a NPM package for a react component. Tried my hands on writing a react radio button component, with webpack and ES6. You can checkout the source code in github repository:  https://github.com/tan31989/react-radio-button  and the npm repository link is:  https://www.npmjs.com/package/react-radio-button . Here is a quick update on the webpack I used for production and deploying the react component as a library so that others can use it: webpack.config.build.js const path = require('path'); const webpack = require('webpack'); module.exports = { entry: [ /** This section is basically the hook for letting the webpack scan your code * to find that right set of files to bundle up for the library you would * want to expose. In my case, the file inside src/components is where * I have the file to be provided as a library. * This section is also known as entry point */ './src/components/Radi