Published in JPAL, SA Internal Newsletter, September 2018
As I move on from J-PAL for doctoral studies, here is a reflection on my journey as a Senior Research Associate on Mindspark Rajasthan, written in the form of a message across time to my younger self.
Dear Younger Self,
The field is exhausting. Those unending vendor and staff calls, all the fires that you put out, all the PI calls that you brave at the end of a tiring day – you are a hero. Your idealism and commitment to ethics unmarred by practical challenges is worth saluting.
I know somewhere deep inside, you might have developed a sense of detachment from data, particularly on the analysis front. Trust me, it is just a matter of bandwidth. Once you have the time and mental space, this stuff is no hard nut to crack. It is sheer logic and pure joy.
The biggest thing you will learn is that STATA is not as hard as you secretly believed. I know that sitting in a remote field office, battling with so many real challenges, every little error that Stata threw up reinforced your belief that this is not your cup of tea. But just dump that thought. Seriously. The truth is that you will start loving data. It is going to be the coolest thing ever.
Let me give you a sneak peek into your future – two years from now you will be managing datasets which span billions of rows. These are system-generated datasets of students using Mindspark software in government schools of Rajasthan. The data is vast, voluminous and generated real time. I have recently discovered and that this is what people call Big Data! You remember those well-annotated do-files you saw your PIs making and editing? Yes, you will soon be writing do-files like those, all by yourself!
Trust me when I tell you that data is beautiful. It seems mysterious at first but as you keep on playing and exploring, it starts unravelling. You start seeing patterns, absence of patterns and surprises all along. You will start discovering hidden stories and solve unsolved mysteries. Your prior field experience will help you connect the dots and make you realise how data is truly a shadow of reality.
Having said that, the process will not be free of challenges. You will receive big dumps of system data to manage and analyse. There will be times when you will feel lost amidst reshaping, merging and appending datasets. There will be times when you will go down a rabbit hole and lose sense of the bigger picture. The journey will be circuitous – involving digressions and meandering pathways with lots of trial and error. Sometimes you will reach a dead-end and bump against a wall.
But the trick here is to keep trying and pushing. Your stubbornness will serve you well. Keep your target in front. In the process, look for ways and commands to reach where you want to. Check out stata help files, google your queries, ask your peers in office – don’t stop until you reach the solution. If at any point it becomes overwhelming, keep a notebook handy. Write the problem down. You will start buzzing with solutions and ideas by the time you have finished writing. Eventually, the walls will come tumbling down and you will emerge stronger with every challenge. The fruit of your labour will shine through every beautiful graph you output for your PI’s slides.
On this journey, you will not be alone. You will have a wise sage as your manager. His name will be Rama. He will demystify all the esoteric jargon and illuminate your mind with astounding clarity.
Brace yourself for the expedition that awaits ahead. You will be so grateful that you stuck the course and followed through.
With Love,
Your Older Self
Dear Younger Self,
The field is exhausting. Those unending vendor and staff calls, all the fires that you put out, all the PI calls that you brave at the end of a tiring day – you are a hero. Your idealism and commitment to ethics unmarred by practical challenges is worth saluting.
I know somewhere deep inside, you might have developed a sense of detachment from data, particularly on the analysis front. Trust me, it is just a matter of bandwidth. Once you have the time and mental space, this stuff is no hard nut to crack. It is sheer logic and pure joy.
The biggest thing you will learn is that STATA is not as hard as you secretly believed. I know that sitting in a remote field office, battling with so many real challenges, every little error that Stata threw up reinforced your belief that this is not your cup of tea. But just dump that thought. Seriously. The truth is that you will start loving data. It is going to be the coolest thing ever.
Let me give you a sneak peek into your future – two years from now you will be managing datasets which span billions of rows. These are system-generated datasets of students using Mindspark software in government schools of Rajasthan. The data is vast, voluminous and generated real time. I have recently discovered and that this is what people call Big Data! You remember those well-annotated do-files you saw your PIs making and editing? Yes, you will soon be writing do-files like those, all by yourself!
Trust me when I tell you that data is beautiful. It seems mysterious at first but as you keep on playing and exploring, it starts unravelling. You start seeing patterns, absence of patterns and surprises all along. You will start discovering hidden stories and solve unsolved mysteries. Your prior field experience will help you connect the dots and make you realise how data is truly a shadow of reality.
Having said that, the process will not be free of challenges. You will receive big dumps of system data to manage and analyse. There will be times when you will feel lost amidst reshaping, merging and appending datasets. There will be times when you will go down a rabbit hole and lose sense of the bigger picture. The journey will be circuitous – involving digressions and meandering pathways with lots of trial and error. Sometimes you will reach a dead-end and bump against a wall.
But the trick here is to keep trying and pushing. Your stubbornness will serve you well. Keep your target in front. In the process, look for ways and commands to reach where you want to. Check out stata help files, google your queries, ask your peers in office – don’t stop until you reach the solution. If at any point it becomes overwhelming, keep a notebook handy. Write the problem down. You will start buzzing with solutions and ideas by the time you have finished writing. Eventually, the walls will come tumbling down and you will emerge stronger with every challenge. The fruit of your labour will shine through every beautiful graph you output for your PI’s slides.
On this journey, you will not be alone. You will have a wise sage as your manager. His name will be Rama. He will demystify all the esoteric jargon and illuminate your mind with astounding clarity.
Brace yourself for the expedition that awaits ahead. You will be so grateful that you stuck the course and followed through.
With Love,
Your Older Self
