[{"content":"Here\u0026rsquo;s a pattern I keep seeing: an AI project looks most impressive right before anyone really understands it.\nThe demo has one job — make people believe. One input, one output, one \u0026ldquo;wow\u0026rdquo; moment. Feed it a messy PDF, watch it spit out clean data. Ask it a question, get a smart answer. Everyone in the room leans in.\nThen someone tries to ship it. And the cracks show up fast.\nWhat actually happens after the demo Say you built a bot that reads invoices and pulls out the total, vendor, and due date. In the demo, you tested it on ten clean PDFs. It nailed all ten.\nThree weeks later, in production:\nA scanned invoice comes in sideways. The model reads the total as $0. Two vendors use the same layout but different currency symbols. It picks USD both times. Someone emails a screenshot instead of a PDF. The pipeline chokes. The finance team starts double-checking every output — which defeats the point of automating it. Nothing here means the model is bad. It means the demo never tested for reality. A demo answers \u0026ldquo;can this work once?\u0026rdquo; Production asks \u0026ldquo;how often does this fail, and what happens when it does?\u0026rdquo; Those are different questions, and only one of them tells you if you have a product.\nStop treating the model as the whole product It\u0026rsquo;s easy to think the model is the product. It isn\u0026rsquo;t. It\u0026rsquo;s one part sitting inside a bigger workflow — and the workflow is what the user actually experiences.\nTake a support-ticket triage system. Say the model correctly labels tickets 90% of the time. Sounds great. But the product still falls apart if:\nNobody set a confidence threshold, so uncertain guesses get treated the same as sure ones. There\u0026rsquo;s no way to route a low-confidence ticket to a human. Agents can\u0026rsquo;t correct a wrong label in two clicks — so they just work around the tool. Nobody\u0026rsquo;s watching whether ticket topics have shifted since last month. None of that is a \u0026ldquo;the model needs to be smarter\u0026rdquo; problem. It\u0026rsquo;s a workflow that was never designed to handle the model being wrong sometimes — which it will be.\nSo before picking a model, ask a more boring question: what decision is this actually supposed to improve? If you can\u0026rsquo;t answer that clearly, you\u0026rsquo;ll end up tweaking prompts and comparing benchmarks forever, with no way to know if any of it matters.\nWrite down how it will fail — before it does Normal software fails loudly. A button does nothing. A page 404s. You know something\u0026rsquo;s wrong immediately.\nAI fails quietly. It gives you a confident, fluent, completely wrong answer — and it can look identical to a correct one. That\u0026rsquo;s the dangerous part.\nSo before launch, sit down and actually list the ways this can go wrong. For the invoice bot, that table might look like:\nWhat breaks What it looks like Why it matters What you do about it Wrong number pulled $450 read as $4,500 Someone gets overpaid Flag anything above a threshold for human check Bad scan quality Blurry or rotated PDF Missing data, silent failure Reject it with a clear message instead of guessing Made-up detail Invents a PO number that doesn\u0026rsquo;t exist Breaks trust in the whole tool Cross-check against known records before accepting Unclear request Invoice in a format you\u0026rsquo;ve never seen Garbage output Route to a human, don\u0026rsquo;t force an answer Cost creep Someone uploads a 200-page contract Surprise API bill Cap file size, batch large jobs This isn\u0026rsquo;t paperwork for its own sake — it\u0026rsquo;s the actual design work. Skip it, and you\u0026rsquo;ll discover these failure modes one angry customer at a time, which is the most expensive way to learn anything.\nTest it before you trust it A lot of teams build the thing, eyeball a handful of outputs, decide it \u0026ldquo;seems good,\u0026rdquo; and ship. Then six weeks later nobody can say whether the last change made things better or worse.\nBuild a test set before you polish anything. It doesn\u0026rsquo;t need to be huge — even 30-50 well-chosen examples beats zero. What matters is that it\u0026rsquo;s honest, not flattering. For the invoice bot, that means including:\nA few clean, easy invoices (the happy path — but not the only path) A blurry scan An invoice missing a due date entirely One in a foreign currency One where the answer genuinely isn\u0026rsquo;t in the document Run every model or prompt change against this same set. Now you can actually see if you\u0026rsquo;re improving or just moving the errors around.\nHuman review isn\u0026rsquo;t a failure — it\u0026rsquo;s often the whole point There\u0026rsquo;s a real temptation to chase \u0026ldquo;fully automated\u0026rdquo; because it sounds more impressive. But full automation is rarely the right first goal. Often the better outcome is: the AI does the boring 80%, and a person handles the judgment calls.\nThe question isn\u0026rsquo;t \u0026ldquo;can we remove the human?\u0026rdquo; It\u0026rsquo;s \u0026ldquo;where does a human\u0026rsquo;s judgment actually matter most?\u0026rdquo;\nSometimes that means: the model handles easy cases and hands off anything uncertain. Sometimes it means the model drafts a reply and a person hits approve. Sometimes the model just gathers the evidence and a human still makes the call. None of these are compromises — they\u0026rsquo;re often what makes people actually trust and use the tool.\nThings will change under you — plan for that Even a system that works well on day one will drift. Users find new ways to use it. Document formats change. The vendor updates the underlying model without asking you. What counted as a normal request in January might be rare by June.\nSo don\u0026rsquo;t just watch if the system is running. Watch if it\u0026rsquo;s still useful:\nHow much is it costing per request, and is that changing? How often are people overriding or rejecting its output? Are the inputs it\u0026rsquo;s seeing starting to look different from what you tested? Is it quietly getting slower? Uptime monitoring tells you the lights are on. None of that tells you whether the thing is still doing its job.\nThe better question to ask before you build anything Don\u0026rsquo;t ask \u0026ldquo;can we use AI for this?\u0026rdquo; That question makes every idea sound viable.\nAsk instead: what decision or bottleneck are we actually trying to fix, and how wrong can this system be before it stops being worth it?\nThat single reframe forces the real questions to show up early — what counts as success, what failure looks like, who reviews the edge cases, what you\u0026rsquo;re tracking after launch. Here\u0026rsquo;s a quick gut-check before you take any demo seriously:\nCan you say, in one sentence, what decision this improves? Do you know what a bad output looks like, specifically? Do you have a test set with real, messy examples — not just the easy ones? Is there a fallback when the model doesn\u0026rsquo;t know? Have you decided where a human needs to be in the loop? Do you know what this costs and how fast it responds, under real load? Is someone actually going to watch this after it ships? If most of those are blank, that\u0026rsquo;s fine — it just means you have a good demo, not a product yet. The gap between the two isn\u0026rsquo;t more AI. It\u0026rsquo;s the unglamorous work of deciding what \u0026ldquo;reliable\u0026rdquo; actually means, and building the workflow around the model instead of just trusting the model itself.\n","permalink":"https://romeepanchal.com/posts/ai_engineering/why_ai_projects_fail_after_the_demo/","summary":"A practical look at why impressive AI prototypes often fail in production, and how to reason about the gap between a demo and a dependable system.","title":"Why AI Projects Fail After the Demo"},{"content":" flowchart LR A[\u0026#34;Sequence memory\u0026lt;br/\u0026gt;LSTMs\u0026#34;] --\u0026gt; B[\u0026#34;Contextual meaning\u0026lt;br/\u0026gt;ELMo and BERT\u0026#34;] B --\u0026gt; C[\u0026#34;Parallel scale\u0026lt;br/\u0026gt;Transformers\u0026#34;] C --\u0026gt; D[\u0026#34;General capabilities\u0026lt;br/\u0026gt;T5 and GPT\u0026#34;] D --\u0026gt; E[\u0026#34;Human alignment\u0026lt;br/\u0026gt;Instruction tuning and RLHF\u0026#34;] The recurring pattern behind modern NLP progress Innovation as a Relay Race#Every landmark in modern NLP began as a modest answer to a concrete limitation. One group publishes a clever fix, another notices the remaining crack, and a third discovers a shortcut the first two never imagined. LSTMs rescued RNNs from vanishing gradients; contextual embeddings made static vectors obsolete; transformers smashed the parallel-computing ceiling; instruction-tuned models closed the human-alignment gap.\nTracking that relay race is more instructive than memorising any single result: it shows why each breakthrough mattered and how the field’s centre of gravity kept shifting.\nBelow is a narrative — rather than a checklist — of the papers that turned yesterday’s impossibilities into today’s defaults.\nGlobal and Contextual Word Representation#GloVe (2014) argued that local window-based prediction (word2vec) ignores global statistics, so it factorises a log-co-occurrence matrix instead. The jump in analogy accuracy hinted that richer context matters.\nFour years later, ELMo (2018) showed an even bigger flaw: static embeddings cannot capture homonyms or polysemy. By feeding sentences through a bi-directional LSTM language model and taking layer-weighted mixtures, ELMo made word meaning conditional on the whole sequence — fuel for the first double-digit gains across QA and coreference tasks.\nThat insight reached escape velocity with BERT (2018). Replace RNNs with Transformers, mask 15 % of tokens, predict them using bidirectional context, and you have a universal encoder that can be fine-tuned for almost anything. RoBERTa (2019) then asked a heretical question: What if BERT’s magic is just large batches, more data, and longer training? By stripping away Next-Sentence Prediction and scaling aggressively, it proved that optimisation tricks sometimes trump architecture tweaks.\nRemembering More Than a Page#As soon as Transformers won short-sequence tasks, researchers hit the quadratic wall: self-attention’s cost explodes with length. Transformer-XL (2019) revived recurrence — storing hidden states from the previous segment and using a relative positional scheme so that textual horizons could extend from 512 tokens to thousands. The leap mattered for story generation and long-form language modelling.\nWhile Transformer-XL kept the full attention pattern, others chased sparsity. Linformer (2020) projected keys and values to a low-rank space, shrinking complexity from O(n²) to O(nk); Longformer (2020) combined sliding-window locality with a few global tokens to process 8 k-token scientific papers on a single GPU. By 2022, Tay et al.’s survey catalogued more than 50 such variants, signalling consensus that hardware — not theory — is now the bottleneck.\nUnifying Tasks and Upsizing Everything#If context length was one ceiling, task-specific heads were another. T5 (2020) demolished both by casting all problems — translation, summarisation, classification — as “input text -\u003e output text.” With a cleaned 750 GB corpus and a seq-to-seq Transformer, it showed that careful task formatting can unlock as much transfer as bigger models.\nSpeaking of bigger: GPT-3 (2020) asked what happens when you simply scale an autoregressive Transformer to 175 B parameters and skip fine-tuning. The answer was in-context learning — the ability to perform new tasks from a few demonstration prompts. Kaplan et al. (2020) then put numbers on the intuition, deriving power-law scaling rules that still guide GPU-budget spreadsheets at every lab.\nAligning Models with Humans#Raw capability is useless — or dangerous — if a model ignores human intent. Ouyang et al. (2022) introduced the now-canonical RLHF pipeline: supervised instruction tuning, a reward model trained on preference comparisons, and PPO reinforcement learning to steer generation. That recipe underpins ChatGPT and every serious deployment concerned with safety, helpfulness, or brand tone.\nEmerging Frontiers#With text well-tamed, attention is turning to the edges: multimodality (CLIP, 2021), 100 k-token memory via state-space models (S4, 2021), retrieval-augmented generation for factual reliability (RETRO, 2022), and sparse mixtures-of-experts to grow capacity without quadratic FLOPs (Switch Transformer, 2021). Each frontier picks up where the last breakthrough shows strain — proof that the relay race continues.\nThen there is a whole new race of reasoning models.\nReasoning-first models — OpenAI o1 / o3. Instead of squeezing ever more tokens into a single forward pass, these variants slow the clock and explicitly allocate extra computation to step-by-step reasoning, tool use and self-reflection. The result is higher accuracy on multi-hop problems and code synthesis without a parameter-count arms race. DataCampTechTargetNext-gen Mixture-of-Experts — DeepSeek-R1 (671 B total / 37 B active). DeepSeek shows MoE is no longer an exotic Google-only trick: by activating a tiny slice of the network per token, it achieves GPT-4-class math-and-coding scores while running on commodity H800s — an order-of-magnitude cut in training FLOPs. HeidloffModularThe Wall Street JournalOpen-source scale — LLaMA 4 (April 2025). Meta’s fourth-generation family adds two open-weights variants (Maverick \u0026 Scout) that match GPT-4-level benchmarks while remaining fine-tune-friendly. For academics and start-ups, it resets the “free-to-tinker” baseline. AiWireWikipediaUltra-long context — GPT-4.1 (up to 1 M tokens). The new flagship pushes context length from 128 k to a million tokens, enough to hold eight copies of the entire React codebase or a week of Slack history. It also closes the coding-accuracy gap with domain-specific copilots. OpenAITechTargetReutersVision-enhanced alignment — Claude 3.5 Sonnet. An thropic’s mid-sized model leap-frogs its own Claude 3 Opus on vision and reasoning tasks, accurately reading charts, receipts and low-quality images — critical for retail and logistics workflows. AnthropicDataCampEach effort tackles the pressure points revealed by the previous wave. These additions reinforce the same relay pattern we’ve seen:\nIdentify a practical ceiling (reasoning depth, training cost, openness, context window, multimodal grounding).Engineer a targeted workaround (scheduled reflection, sparse experts, permissive licensing, memory-efficient attention, vision encoders).Watch the workaround create its follow-up questions (how to debug hidden experts? how to search a million-token prompt?).Conclusion — What This Journey Teaches#Progress is iterative but not incremental. A single clever hack (masking, recurrence, RLHF) can open a new design space.\nBottlenecks create research themes. Once representation quality plateaued, context length and compute cost became the urgent problems; now, alignment and grounding dominate.\nAdopt selectively. Map your pain point — representation, context, scale, or alignment — to the paper that solved it. Implement that solution before chasing the next hype wave.\nReferences#Hochreiter, S., \u0026 Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780.\nPennington, J., Socher, R., \u0026 Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. EMNLP.\nPeters, M. E., et al. (2018). Deep Contextualized Word Representations. NAACL.\nDevlin, J., et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL.\nLiu, Y., et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692.\nDai, Z., et al. (2019). Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. ACL.\nWang, S., et al. (2020). Linformer: Self-Attention with Linear Complexity. arXiv:2006.04768.\nBeltagy, I., Peters, M., \u0026 Cohan, A. (2020). Longformer: The Long-Document Transformer. arXiv:2004.05150.\nRaffel, C., et al. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR, 21(140).\nBrown, T. B., et al. (2020). Language Models are Few-Shot Learners. NeurIPS.\nKaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.\nOuyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. arXiv:2203.02155.\nRadford, A., et al. (2021). Learning Transferable Visual Models from Natural Language Supervision. ICML.\nGu, A., et al. (2021). Efficiently Modeling Long Sequences with Structured State Spaces. arXiv:2111.00396.\nFedus, W., et al. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. JMLR, 23(120).\nBorgeaud, S., et al. (2022). Improving Language Models by Retrieving from Trillions of Tokens. Nature, 601, 590–595.\nTay, Y., et al. (2022). Efficient Transformers: A Survey. ACM Computing Surveys, 55(6), 1–28.\nHeidloff, N. (2025). Key Concepts of DeepSeek-R1. Heidloff\nModular AI. (2025). Exploring DeepSeek-R1’s Mixture-of-Experts Architecture. Modular\nMeta AI. (2025). LLaMA 4 release blog \u0026 documentation. AiWire\nOpenAI. (2025). Introducing GPT-4.1 in the API. OpenAI\nReuters. (2025, Apr 14). OpenAI launches GPT-4.1 models with improved coding \u0026 long context. Reuters\nAnthropic. (2024). Introducing Claude 3.5 Sonnet. Anthropic\n","permalink":"https://romeepanchal.com/posts/deep_learning/nlp_progress/","summary":"Trace how recurrent networks, attention, transformers, instruction tuning, and RLHF built on one another.","title":"From LSTMs to RLHF — How One Idea Ignites the Next"},{"content":"Are you fascinated by deep learning\u0026rsquo;s transformative power but unsure how to navigate the journey from logistic regression to mastering transformer architectures? You’re not alone. Transformers are the backbone of modern AI, power innovations in natural language processing, computer vision, and beyond, but getting there can feel daunting.\nIn this blog, I outline a structured, week-by-week learning path that takes you from the foundational concepts of machine learning to building and fine-tuning your transformer models. Whether you\u0026rsquo;re a beginner or looking to deepen your expertise, this roadmap combines key concepts, curated resources, hands-on projects, and practical tips to make your progress achievable and rewarding.\nHere\u0026rsquo;s a detailed week-by-week learning path. Each week we will build upon previous knowledge:\nWeek 1: Linear Models#Topics:\nLogistic regression (binary classification)Cross-entropy lossSoftmax function for multi-class problemsDeep dive into gradient descent variants (SGD, Mini-batch)Resources:\nhttps://www.youtube.com/playlist?list=PLkDaE6sCZn6FNC6YRfRQc_FbeQrF8BwGI3Blue1Brown Neural Network video seriesArticle: A Visual Explanation of Softmax RegressionProject:\nImplement logistic regression from scratch using NumPy.Use sklearn for logistic and softmax regression on sample datasets.Week 2: Neural Network Foundations#Topics:\nSingle-layer and multi-layer perceptronsActivation functions (ReLU, tanh)Forward and backward propagationDerivation of backpropagationResources\nDeep Learning\" by Ian Goodfellow – Chapter 6 (Feedforward Neural Networks)Stanford CS231n lecture notes on backpropTensorFlow Playground to visualize FFNNshttps://www.sscardapane.it/alice-book/Project\nImplement a basic FFNN from scratch (with one hidden layer).Create a simple feedforward neural network (FFNN) to classify the MNIST digits dataset using a framework like PyTorch or TensorFlow.Experiment with different activation functions (ReLU, sigmoid) and compare performance.Week 3: Deep Neural Networks#Topics:\nMultiple hidden layersAdvanced activation functionsInitialization techniquesBasic optimization algorithms (Momentum, RMSprop)Resources:\nAdam optimizer paperFastAI Deep Learning Course Part 1PyTorch tutorialsNeural Networks and Deep Learning by Michael NielsenProject:\nImage classification on CIFAR-10 with deep neural networkApply gradient descent with different learning rates and optimizers (SGD, Adam).Week 4: Advanced Optimization \u0026 Regularization#Topics:\nBatch normalizationDropoutL1/L2 regularizationLearning rate schedulingResources:\nBatch Normalization paperDropout paperStanford CS224n lecturesProject:\nBuild a deep network for sentiment analysis with regularization techniquesWeek 5: Sequential Data \u0026 RNNs#Topics:\nRNN architectureBackpropagation through timeVanishing/exploding gradientsLSTM cellsResources:\nLSTM paperAndrej Karpathy\u0026rsquo;s blog on RNNsChris Olah\u0026rsquo;s LSTM blog postProject:\nCharacter-level text generation using LSTMWeek 6: Introduction to Attention Mechanisms#Topics:\nEncoder-decoder architectureTeacher forcingBeam searchBasic attention mechanismsResources:\nSeq2Seq paperNeural Machine Translation paperLilian Weng\u0026rsquo;s attention blog postProject:\nImplement Bahdanau (additive) or Luong (multiplicative) attention.Implement a basic sequence-to-sequence model for translating English to French using Bahdanau attention (use small parallel corpora).Week 7: Self-Attention and Multi-Head Attention#Topics:\nScore functionsQuery-Key-Value conceptSelf-attentionDot-product attention vs. additive attentionResources:\nAttention Is All You NeedJay Alammar\u0026rsquo;s blog on attentionhttps://jalammar.github.io/illustrated-transformer/Project:\nManually compute self-attention for a toy example and build a self-attention layer using PyTorch.Extend the implementation to a multi-head attention mechanism and validate its performance on sequence data.Week 8: Multi-Head Attention and Positional Encoding#Topics:\nMulti-head attentionPositional encodings (sinusoidal functions)Resources:\nAttention Is All You Needhttps://jalammar.github.io/illustrated-transformer/https://towardsdatascience.com/transformers-explained-visually-part-3-multi-head-attention-deep-dive-1c1ff1024853https://medium.com/@sayedebad.777/building-a-transformer-from-scratch-a-step-by-step-guide-a3df0aeb7c9aProject:\nWrite code for multi-head attention.Implement positional encoding and visualize it.Implement a custom multi-head attention module and add positional encodings. Use this to classify sequences of text (e.g., positive/negative sentiment).Week 9: Transformers Block (Encoder-Decoder Structure)#Topics:\nEncoder and decoder architectureResidual connections and layer normalizationResources:\nhttps://nlp.seas.harvard.edu/annotated-transformer/https://www.datacamp.com/tutorial/an-introduction-to-using-transformers-and-hugging-faceProjects:\nBuild a simple transformer encoder layer.Build a transformer encoder for a language modeling task using PyTorch or TensorFlow.Train the encoder on a small text dataset (e.g., Shakespeare sonnets).Week 10: Full Transformer Model#Topics:\nEnd-to-end implementation of the original transformerComplete transformer architectureResources:\nhttps://arxiv.org/abs/1706.03762https://nlp.seas.harvard.edu/annotated-transformer/https://www.datacamp.com/tutorial/an-introduction-to-using-transformers-and-hugging-faceProjects:\nImplement a transformer-based sequence classification task.Implement a simplified transformer model from scratch and apply it to text summarization or machine translation.Use performance metrics (BLEU score for translation, ROUGE score for summarization) to evaluate results.Week 11: Transformer Variants (BERT, GPT)#Topics:\nBERT (masked language modeling)GPT (causal language modeling)Resources:\nhttps://arxiv.org/abs/1810.04805https://arxiv.org/abs/2005.14165https://huggingface.co/learn/nlp-course/en/chapter4/2Projects:\nFine-tune a pre-trained BERT or GPT model using HuggingFace.Implement a chatbot using a GPT model for conversational responses.Additional Project Ideas#Once you complete the core projects, reinforce your learning with larger, integrative projects:\nSentiment Analysis on Movie Reviews: Use transformers for sentiment classification on the IMDB dataset.Named Entity Recognition (NER): Implement NER using transformers and fine-tune on the CoNLL-2003 dataset.Question Answering System: Use BERT or RoBERTa to create a question-answering application on a custom dataset.Week 12: Advanced Techniques and Optimization#Topics:\nModel distillationReducing memory consumption (efficient transformers)Resources:\nhttps://arxiv.org/abs/2009.06732https://arxiv.org/abs/2001.04451https://huggingface.co/docs/transformers/en/trainingProjects:\nExperiment with efficient transformer architectures (e.g., Reformer or Longformer) for a custom dataset with long sequences.Apply model distillation to compress a large transformer model into a smaller, faster one for inference.Happy learning!\nGet the latest articles on AI delivered straight to your inbox. Subscribe here!\n","permalink":"https://romeepanchal.com/posts/deep_learning/learning_path/","summary":"A concept-first path from logistic regression through neural networks, attention, and modern transformers.","title":"From Logistic Regression to Transformers: Learning Path"},{"content":"This world is full of ambitious people who are lifelong learners. They pursue knowledge even while managing careers or businesses, whether to satisfy their curiosity or to enhance their skills.\nI am a lifelong learner, though perhaps not among the highly ambitious. Learning is my passion - I\u0026rsquo;m drawn to newer technologies and understanding more about the world. I am a full-time machine learning engineer. If you follow AI news, you know a lot is happening in this field. Even keeping up with a single sub-field is challenging. Yet, I consistently try to learn about new methods and technologies that could create value in my work. To do so I need to find a dedicated time to self-study. In this post, I will share my experience with self-learning.\nBut first things, first#What do I mean by self-learning? Does reading articles count as self-learning? When I say self-learning, I emphasize learning a full-length course. For example, if someone wants to learn machine learning, which was not part of their curriculum, they spend hundreds of hours on it. They need to study fundamental subjects, like underlying mathematics. They need to understand and work through the ML algorithms. This requires focus and prioritization.\nMy experience with self-study#Why I wanted to learn new things when I already had a job?#I started working as a software engineer right after engineering college. There, I was mostly responsible for writing Perl scripts for data validation. Even though I like coding a lot, I did not enjoy working on plain scripting without knowing what was going on and what the big picture was. I wanted to solve some interesting problems. Thankfully, I came across the term machine learning and found it very interesting. I read one article where people called data science the sexiest job of the 21st century. I thought this could be something I was looking for. This could be a good career option for me. So, I decided to learn this new subject.\nHow I started it?#I searched for the required skillset to become a data scientist. At that time, only a few resources were available to know how to begin with machine learning. However, whatever blog posts and newsletters I read, they all pointed out that first, I needed to learn a solid mathematical foundation, such as linear algebra, multivariable calculus, probability, and statistics. Then, I should learn machine learning. This required serious upskilling.\nI started gathering the required material, such as mathematics and programming books. I surmised I could learn programming, math, and machine learning simultaneously. I was excited in the beginning, as I was learning something new.\nIn the first few weeks, I followed a schedule that I created. But soon I started skipping some of my subjects. Or, reading what I had already covered in the past. I was doing some activities like looking for other material to learn or skipping ahead in the book to see what else was coming my way. I realized I wasn\u0026rsquo;t making much progress. And I started wondering why.\nMy failures#When I look back, one thing is clear I tried to do too many things at once, making it hard to maintain focus and make real progress. I often jumped between different subjects without mastering any one topic. Switched between multiple learning resources rather than following one comprehensive source. I failed to establish a consistent schedule I could stick to. And, I got overwhelmed by different learning formats (videos, books, tutorials) instead of finding what works best for me.\nWhen a person has too many things to do they worry more about the things to do than actually doing them. Without spending quality time on each thing leads to poor results and slow progress.\nAnother realization was that with so many tasks at hand, there was limited time for rest, which resulted in physical exhaustion.\nOther difficulties with self-study#There are other difficulties as well such as you have family obligations to fulfill. You have a full-time job which requires your attention. Sometimes you have to travel. You need to give time to your family. You have to do laundry, groceries and other chores. You need to look over repairs and other household stuff. And after doing all this you become exhausted. You have limited physical and mental energy left to do anything.\nMy learnings about self-learning#There are a few basic problems:\nI tried to do too many things at once. I have to accept that I can not do all things at once. I have to take things slowly so that I can make steady progress. I should focus on quality rather than quantity.Jumping from one learning resource to another without making much progress.Lack of habit of studying regularly.What worked for me?#Once I realized these problems, I addressed them one by one.\nThe first problem was studying multiple subjects in parallel. A full time does not allow much free time. I had to prioritize this time for studying, entertainment, family and rest. Given the constraints, I found that I could only focus on one subject per week.Next in the line was to fix the problem of hopping from one resource to another. Since I learn best from books. I decided to stick to one book. To make it even more sticky, I decided to set small goals. I read somewhere that setting specific goals can help achieve them. It was the time to test that. I started setting one or two topics for a week time and focused on completing just that. With this strategy, soon I realized that I made a lot of progress in a short amount of time. This step was a particular game-changer for me.The next important bit was to build a learning habit. With my experiments with self-study, one thing I can say for sure is that if anyone wants to make steady progress, they have to show up consistently. For this I tried the following golden rule of habit building - Never miss your learning schedule twice in a row. By the way, this came from James Clears in Atomic Habits, not my idea.By addressing these elements systematically, I transformed my learning journey from scattered efforts to steady progress. The key was not just knowing what to change, but implementing these changes gradually and consistently.\nRemember, effective self-learning isn\u0026rsquo;t about speed - it\u0026rsquo;s about sustainable progress and genuine understanding. Start small, stay consistent, and adjust your approach as needed. Your learning journey is unique, and finding what works best for your situation is part of the process.\nHappy learning!\nGet the latest articles on AI delivered straight to your inbox. Subscribe here!\n","permalink":"https://romeepanchal.com/posts/life/learnings/self_study_experience/","summary":"Lessons from the frustrations, false starts, and breakthroughs that come with learning independently.","title":"Learning on Your Own: Challenges, Failures, and Successes"},{"content":"Our habits shape us. Some habits can make us productive, others can drag us and won’t let us achieve anything. That is why it is important that we don’t let our bad habits hold us back in life.\nWhy do we have habits?#Human brain is an incredibly complex evolved machine. It tries to optimize processes to keep everything running smoothly. When we attempt to do a new task, it takes a lot of our attention, and thus, require a lot of brain power. Every time we repeat that task, our brain tries to optimize the process. If that task is repeated enough time, brain will create a fixed optimized algorithm to perform that task so that it doesn’t need to spend more energy in thinking. This fixed optimized algorithm is what we call a habit.\nWhy do we face difficulty while forming a new habit?#As I already mentioned, when we attempt to do a new task, it takes a lot of our attention, and thus, require a lot of brain power. Brain tries to keep the energy consumption to low level. To do so it chooses an easy route i.e. a preexisting habit. So, next time when we decide to repeat that task again to make it into a habit, our brain triggers an automatic subconscious cycle to do what we used to do at that time.\nAnother reason is emotional. When we do a task in which we take pleasure, our body releases a chemical called dopamine. So, when you pull out your smartphone out of habit, your body has taken over your mind, and it is seeking the dopamine chemicals that it has become habituated to.\nThat is why habit forming process is difficult.\nWhat should we do to develop a new habit?#People often tries to develop new habit using willpower. Willpower helps us in getting started. But it does not help us to keep that on repeating again and again.\nSo, what is better way then? What we learnt about habits is that “decision” and “emotional need” are two things that make it difficult to develop new habit. In order to develop a new habit, we should remove these difficulties.\nThere are two simple things we can do to develop:\nDesign. Make an easier environment for new habit so that brain does not waste energy in deciding whether to do that task or stick to subconscious cycle. Suppose you want to study after dinner. For this you need to prepare the environment in advance. Keep study material ready and keep away things that you usually involve with at that time like laptop or smartphone etc. This way you don’t need to take a decision every time.\nReward. Use positive reinforcement. Every time you stick to the habit you want to develop, give yourself a reward. It can be anything that you feel like a reward. This way you are attaching positive emotions to that habit.\nIn addition to above practice, I also suggest reading following books:\nThe Power of Habit — Charles Duhigg.Atomic Habits - James ClearStay Positive :)\n","permalink":"https://romeepanchal.com/posts/life/magic_of_habits/","summary":"How small repeated behaviors compound into meaningful change, and how to design habits that last.","title":"The Magic of Habits"},{"content":"One of the most important facts to understand in life is that you don\u0026rsquo;t always get what you want. You can hope to achieve things in life. You can even work harder than others. However, there are factors beyond our control that will affect the overall outcome. Those factors play a significant role in our life.\nBut why am I writing about this? I think, when life goes smoothly, we rarely question things. We feel content, and everything seems to be going according to plan. Why worry at such a time, right? But as soon as things do not go according to our plan, then we realise the more realistic world around us. We suddenly see things from a philosophical angle.\nRecently, I was thinking that all the things in the world were going nicely. Then BAM. You get a reality check. You get a reminder. Nothing is permanent. Even if you work your ass off towards something, if you are unlucky, you may not get the desired results.\nThis realisation seems negative at first, doesn\u0026rsquo;t it? Seems like things can go south at the most crucial time of your life and what you worked for so long will completely go to waste. But that is not the case. I once read that with determination (and a positive outlook), you can find light even in the darkest hour. If nothing is permanent in life then the same applies to our bad fate. It will also change one day and come to work with us. That day will be awesome. Eventually, circumstances will align in our favour, and our efforts will yield results.\nSo, what\u0026rsquo;s my takeaway from this realisation?\nFirst of all, that writing helps a lot. Before writing this article, I was in a questioning mode. But as I was writing this article, things started to become a little bit clear. Which is a big mental shift by the way.\nSecond, nothing is permanent in life. It applies both ways. When things are good and when things are bad. This understanding offers hope in bad times and fosters gratefulness in the good times.\nThird, luck plays a big role in life. However, this does not mean we should stop making efforts. In fact, use it to your advantage. You never know when good luck will knock on your door. Keep putting in efforts and be ready for that moment.\nIn conclusion, life\u0026rsquo;s unpredictability can be both challenging and exciting. By acknowledging this reality and maintaining a balanced perspective, we can navigate life\u0026rsquo;s ups and downs with resilience and optimism.\n","permalink":"https://romeepanchal.com/posts/life/finding_hope_in_bad_times/","summary":"A personal reflection on uncertainty, resilience, and finding room for hope when plans fall apart.","title":"Finding Light in Unpredictability"},{"content":"In recent years, machine learning (ML) has rapidly advanced and become a key component of modern technology. Whether applied to recommendation systems, autonomous driving, healthcare, or finance, machine learning’s influence is undeniable. Many practitioners in the field come from diverse backgrounds, sometimes with limited formal mathematical training but strong programming and data manipulation skills. While this might be sufficient to implement machine learning models using popular libraries like TensorFlow, PyTorch, or scikit-learn, there is a compelling case for deepening one’s understanding of mathematics even when you already have hands-on experience in building models. Mathematics provides the theoretical foundation for machine learning and enables practitioners to elevate their work, enhance model performance, and solve complex problems with greater insight and creativity.\nThis essay will explore several key reasons why learning mathematics is essential for machine learning professionals. It will cover the role of math in understanding model behaviour, optimizing performance, creating new models, avoiding common pitfalls, and effectively communicating ideas.\nUnderstanding Model Behaviour and Inner Workings#Machine learning models, particularly complex ones like deep neural networks, are often treated as black boxes by those who lack mathematical training. With strong libraries available to abstract much of the underlying complexity, it’s possible to build and deploy models without fully understanding how they work. However, this is a risky approach. Without a solid grasp of the mathematical principles that govern these algorithms, you might miss important insights into their behaviour, limitations, and potential failure modes.\nFor example, linear regression is a simple algorithm used frequently in ML. On the surface, it seems straightforward: find the best fit line for a given set of data points. But what does “best fit” mean? How does the algorithm determine this? The answer lies in concepts from calculus (gradient descent) and linear algebra (matrix operations). Understanding these mathematical concepts allows you to grasp how the model makes predictions, why it behaves the way it does, and how to interpret the coefficients and performance metrics meaningfully.\nBy studying the underlying mathematics, you can understand not just how models like neural networks or support vector machines (SVMs) work, but why they behave the way they do under different conditions. This depth of understanding empowers you to make better decisions when tuning models, selecting algorithms, or designing architectures.\nOptimising Model Performance#When building machine learning models, practitioners often focus on improving metrics like accuracy, precision, or recall. Achieving high performance typically involves hyperparameter tuning, feature engineering, and experimenting with different algorithms. While practical experience helps with these tasks, a firm mathematical foundation can give you a more sophisticated toolkit for optimisation.\nFor instance, consider the concept of regularisation, a critical technique used to prevent overfitting. Regularisation adds a penalty term to the loss function, which discourages complex models from fitting the noise in the data. Understanding L1 and L2 regularisation in terms of norms and vector spaces can help you determine when to use them, how to adjust the penalty terms, and why they are effective in certain situations. Without this mathematical insight, regularisation might just seem like an arbitrary trick.\nSimilarly, optimisation algorithms like stochastic gradient descent (SGD), Adam, and RMSprop are at the heart of most modern ML models, but they operate on mathematical principles from calculus and statistics. Knowing how gradients work, why learning rates matter, and how loss surfaces behave gives you greater control over the training process, helping you avoid issues like vanishing or exploding gradients, slow convergence, or getting stuck in local minima.\nCreating and Innovating New Models#While existing machine learning models and algorithms can handle many real-world problems, innovation in the field requires a deep understanding of mathematics. Practitioners who want to go beyond using pre-built models and instead contribute to advancing the state of the art will inevitably need to master the mathematical theories that underlie machine learning.\nFor example, generative models like Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) rely on advanced topics from probability theory, linear algebra, and information theory. To truly innovate with these models, you need to understand concepts like Kullback-Leibler divergence, latent space, and Jensen-Shannon divergence. Armed with this knowledge, you can push the boundaries of current models or even create new ones better suited to specific tasks.\nMoreover, learning mathematics equips you with the tools to break down complex problems and formulate new algorithms from scratch. As the field of machine learning continues to evolve, professionals with strong mathematical expertise are more likely to lead these innovations by designing algorithms that address limitations in existing methods or tackle new challenges altogether.\nAvoiding Common Pitfalls#Machine learning models are often sensitive to the quality of the input data, and incorrect assumptions or overlooked details can lead to flawed results. A solid understanding of statistics, probability, and linear algebra helps practitioners recognize these pitfalls before they become major issues.\nOne common pitfall is overfitting, where a model performs well on training data but poorly on unseen data. Knowing the mathematical foundations behind model complexity and regularization allows you to effectively combat overfitting. Similarly, understanding the bias-variance tradeoff, another core statistical concept, can help you balance model flexibility with generalization.\nAnother example is in handling data. Matrix operations are central to many machine learning algorithms, and errors in understanding matrix dimensions, properties, or decompositions can lead to incorrect implementations. Additionally, probability theory is essential for tasks like estimating uncertainty in predictions, interpreting results, and designing models for tasks involving randomness, like reinforcement learning or Bayesian inference.\nWithout this mathematical foundation, you might miss subtle issues in data processing, model evaluation, or even in the assumptions of the algorithms themselves, leading to inaccurate or misleading results.\nEffective Communication and Collaboration#Finally, mathematics is the universal language of machine learning and data science. Whether you are working in a team of engineers, collaborating with researchers, or presenting your work to stakeholders, the ability to communicate the principles behind your models is invaluable. Even though many people can understand the output of a model (such as a prediction or classification), fewer can appreciate the nuances of how that output was achieved.\nBy developing a solid understanding of the mathematical principles that govern machine learning models, you can effectively explain your choices, justify trade-offs, and collaborate with others who are more mathematically inclined. This skill is particularly important in interdisciplinary teams where data scientists, engineers, statisticians, and domain experts work together. Clear communication based on a shared understanding of mathematical principles ensures that the entire team can align on model goals, limitations, and capabilities.\nConclusion#While it is possible to build machine learning models with a basic understanding of mathematics, a deeper knowledge of mathematical principles offers numerous advantages. It allows you to understand how models work, optimise their performance, create new algorithms, avoid common pitfalls, and communicate effectively with others in the field. As machine learning continues to evolve, those with strong mathematical foundations will be better positioned to innovate, solve complex problems, and contribute meaningfully to the advancement of the field. Therefore, investing in learning mathematics is a crucial step for anyone serious about mastering machine learning.\n","permalink":"https://romeepanchal.com/posts/general/why_learn_maths/","summary":"How mathematical foundations improve model intuition, debugging, optimization, and technical communication.","title":"Why Should You Learn Mathematics for ML"},{"content":"Data science is one of the fastest-growing and most exciting fields today. It offers a unique blend of challenge and reward, combining the analytical skills of a mathematician with the creativity of a software engineer, all while providing valuable insights for businesses. The field encompasses a variety of roles, including data scientist, data analyst, machine learning engineer, and AI engineer.\nFor those new to the field, data science internships offer a valuable first step, providing hands-on experience with real-world data. If you already have industry experience, building a strong data science portfolio is crucial before applying for jobs. This article will guide you through the practical steps of creating a portfolio to help you land your dream data science job.\nIdentifying Your Area of Interest#To begin your data science journey, it is essential to identify your strengths and your area of interest. Reflect on your experience, projects, and coursework to determine the aspect of data science that resonates with you the most. Ask yourself the following:\nWhat topic or domain in data science fascinates you, such as data analysis, machine learning, or data visualisation?What kind of problems do you want to solve in healthcare, finance, or other fields using NLP, image processing, or general machine learning techniques?Read and research about your area of interest. Talk to people to understand their work and then make an informed choice.\nAssessing Your Skills#Before applying for data science roles, it\u0026rsquo;s crucial to assess your current skills and strengths. Here are some key areas:\nProgramming: Data scientists need to write functional code to explore data, train models, and present results. Python is a popular choice.Data Analysis: A data scientist must be able to clean, manipulate, and analyse data. Statistical concepts and visualisation methods are essential tools for data exploration.Machine Learning: Understanding machine learning and building predictive models is crucial. While it may be tempting to start with models directly, a solid foundation in the mathematical underpinnings of machine learning algorithms allows for a deeper understanding and better long-term results.Communication: Data scientists need to effectively communicate their findings to both technical and non-technical audiences.Once you\u0026rsquo;ve evaluated your skills, identify areas for improvement and focus on building a strong foundation before applying for data science jobs. By taking this proactive approach, you\u0026rsquo;ll be well-positioned to impress potential employers.\nBuilding a Solid Foundation#Coursework, Degrees, and Certifications#Identifying the gap between your current skillset and data science roles is crucial. To fill these gaps, consider enrolling in relevant coursework, pursuing a degree, or obtaining a certification program in machine learning and data science. Carefully evaluate the program\u0026rsquo;s curriculum to ensure it aligns with your interests and career goals.\nParticipating in Data Science Competitions and Hackathons#Data science competitions and hackathons offer a valuable way to test your skills and gain practical experience. These events allow you to learn from peers, build your network, and showcase your abilities to potential employers.\nMy personal experience: I started my career as a software engineer. When I decided to transition to data science, I actively participated in machine learning hackathons. From personal experience, I can tell you that such competitions are a fantastic way to learn new things from your peers, build a network in the field, and grab the attention of potential hiring managers.\nDeveloping a Portfolio#You work hard to acquire the necessary skills of a data scientist. Now, it is time to put those skills to work to create something concrete. One of the best ways to gain experience is through personal projects. Find some real-world data to work on. You can easily find such data from sites like Kaggle.com. Take the data you find interesting and start exploring. Understanding data well is of utmost importance for a data scientist. Once you have explored the data, think of some machine learning use cases that you can build using this data and create a machine learning solution. For example, you can find users’ tweet data from the internet and build a user sentiment prediction model using such data.\nOnce you have a few machine learning projects, create an online portfolio. You can build such a portfolio using GitHub Pages. It is free to use and easy to set up. This also showcases your skill to work with version control tools such as GitHub. Include the links to your projects, the data sources used, and the outcome of the projects. If possible, add links to some data visualisations as they show your skill in working with data. Take some time to write the details of each of your projects.\nThis will show your future employer your ability to frame problems, code, analyse, and build models. Your ability to craft good stories about your projects will also showcase your communication skills.\nBelow are the steps:\nFind Real-World Data: Go to Kaggle choose a dataset that piques your interest and delve into it.Explore and Experiment: Load the data and familiarise yourself with the data, its structure, and the features/variables it contains. Review metadata, data dictionaries, and any documentation provided. Check for missing values, duplicates, outliers, and inconsistencies in the data. Identify any potential data quality issues. Analyze each feature/variable individually. Generate summary statistics, visualize distributions, and identify patterns. Create features and develop a model.Showcase Your Work: Once you have 3-4 projects, create an online portfolio using platforms like Github Pages.Content is King: Include clear descriptions of each project, detailing the data sources used, the methodology applied, and the outcome.Visual Appeal: Incorporate data visualisations where relevant to showcase your data visualisation skills.Storytelling Matters: Take time to craft compelling narratives around your projects. This highlights your problem-solving abilities and communication skills, both valuable assets for data scientists.Creating a Compelling Resume#Your resume is the first impression you make on a potential employer, so it\u0026rsquo;s crucial to create a compelling resume that showcases your qualifications.\nGather Your Content: Reflect on your education, skills, projects (including personal projects!), hackathon experiences, and any other relevant achievements.Targeted Tailoring: Use online resources to find a resume template that complements your experience level and the type of roles you\u0026rsquo;re targeting.Keyword Optimisation: Carefully review job descriptions and incorporate relevant keywords throughout your resume, especially in your skills section. This can help your resume pass applicant tracking systems (ATS) used by many companies.One Strong Base: Instead of creating several entirely different resumes, develop a strong base resume that highlights your core skills and experiences.Targeted Tweaks: When applying for specific jobs, customise your base resume by emphasizing the skills and experiences most relevant to the particular role. This targeted approach demonstrates that you understand the position\u0026rsquo;s requirements and strengthens your candidacy.By following these tips, you can create a resume that effectively showcases your qualifications and increases your chances of landing your dream data science job.\nNetworking and Personal Branding#Word of mouth is a powerful marketing strategy. If a recruiter receives two applications for a role, one through internal referral and the other through an online application, chances are the one that came through internal referral will be given priority. This is why it is crucial to build strong connections.\nYou can utilise the power of your network to fast-track your application process. If someone in your network works at a company you are interested in applying to, you can ask them to refer your profile to the hiring manager.\nThere are several ways to build a strong network:\nAttending Conferences, Meetups, or Events: This is an excellent way to connect with people from various industries. You can find such events online. Attend these gatherings and engage in conversations. Before asking for opportunities, share your interests and what you can contribute.Engaging in Online Data Science Communities: Another effective method is to participate in online communities related to data science. Post questions, provide answers to others\u0026rsquo; queries, comment, and like posts. This not only demonstrates your interest in data science but also increases the likelihood of learning useful techniques from others.Utilising LinkedIn: Make an effort to connect with professionals on LinkedIn. Send personalised messages when connecting. Share your knowledge and remain active on the platform. This way, people will start noticing your skills, and you may receive job opportunities through these connections.By following these tips, you can build a strong network and personal brand that will help you advance your data science career.\nSearching and Applying for Jobs#Once you feel comfortable with the fundamentals, start searching for relevant jobs in the market. Begin by researching companies and organisations that offer data science or machine learning positions. Shortlist the companies that align well with your interests in the field of data science or artificial intelligence. You can visit their career pages and apply directly.\nAnother way to search and apply for jobs is through career portals. Platforms such as LinkedIn, Naukri, and Indeed usually have jobs posted directly by company recruiters. It\u0026rsquo;s quite easy to search for jobs on these portals by applying filters and using specific keywords.\nA third way to apply for a job is through referrals. This is where your networking will come in handy. You can ask your connections or online community members to refer you for roles that interest you.\nPreparing for Interviews#Each company has its selection criteria. Regardless of the number of rounds or exact interview format, all hiring managers look for the following qualities in a candidate:\nTechnical Skills - Familiarity with particular technologies to do the job effectively.Problem-Solving Skills - Ability to solve complex problems with minimal guidance.Communication Skills - Ability to communicate your ideas effectively with other team members and stakeholders.Let\u0026rsquo;s look into each of these areas:\nTechnical Skills#For technical skills, there are two main aspects: 1) the tech stack and 2) conceptual understanding.\nHiring managers are interested in knowing your comfort level with the tech stack they work with. They may judge you based on your proficiency with particular software packages. For example, if you\u0026rsquo;re applying for an AI engineer role, you need to know deep learning frameworks like PyTorch or TensorFlow. For machine learning, you need to know Scikit-learn, NumPy, Pandas, etc. You may be asked questions related to the usage of these technologies. So, check the tech stack used for the role you\u0026rsquo;re applying for and prepare accordingly.\nThe second aspect you may be judged on is conceptual understanding. Interviewers may ask questions to gauge your comfort level with particular topics. For example, for a data science role, they might ask questions related to machine learning, probability, statistics, or mathematics.\nThe ability to code is also very important for data science and machine learning roles. Hence, it\u0026rsquo;s crucial to know the fundamental topics of your programming language. For example, in Python, you need to know about lists, tuples, dictionaries, OOP concepts, iterators, and generators.\nProblem-Solving Skills#Hiring managers want to recruit people who can solve complex problems. To check problem-solving skills, different companies interview candidates in various ways. Typically, they conduct one or more rounds of coding and algorithmic problem-solving interviews. Based on the seniority levels, they may also conduct case study and/or system design interviews.\nPrepare for coding problem-solving at least a month before. You can practice on platforms like LeetCode or HackerRank.\nSimilarly, prepare for case studies or design interviews. You usually learn these skills while working. There\u0026rsquo;s no shortcut to improving these skills. However, you can read examples of case studies and system designs for many different problems.\nCommunication Skills#Finally, communication skills are equally important. Even if a candidate has all the technical knowledge required for a role, they might be rejected if they fail to communicate clearly with the interviewers. Within companies, projects are team efforts. You need to communicate regularly within your team as well as with other teams. In a data science role, you often need to present your model\u0026rsquo;s output to stakeholders. Hence, communication skills are crucial.\nInterviewers don\u0026rsquo;t typically have a round focused solely on communication skills. Instead, they evaluate your communication skills throughout the entire interview process.\nFor experienced candidates, hiring managers like to know about your past projects. They want to know what you did in your previous company, what situations you faced, and how you tackled them. They also like to know the value or impact you created. So, practice how you will present your previous projects.\nAdditional Tips#Mock interviews: Practice with peers or mentorsTime management: Solve problems under timed conditionsWhiteboard practice: Explain solutions visuallyVerbalise thought process: Practice thinking aloudPrepare questions: Show genuine interest in the roleReview your own projects: Be ready to discuss in depthDevelop a problem-solving framework: Approach questions systematicallyPractice explaining complex concepts simplyNegotiating Job Offers#You\u0026rsquo;ve successfully navigated the application process and received a job offer. However, your journey isn\u0026rsquo;t quite over. Negotiating your job offer is a crucial step that can significantly impact your career trajectory and job satisfaction.\nBefore entering negotiations, arm yourself with knowledge. Research industry standards for data science salaries in your area using resources like Glassdoor, PayScale, or the annual Stack Overflow Developer Survey. This information will provide a solid foundation for your negotiation strategy.\nRemember, negotiation isn\u0026rsquo;t just about salary. Consider the entire compensation package, including benefits, bonuses, stock options, remote work flexibility, and professional development opportunities. A lower base salary might be offset by excellent benefits or growth potential.\nWhen you begin negotiations, approach the conversation as a collaborative discussion rather than an adversarial process. Articulate your value proposition clearly, highlighting your unique skills, experience, and potential contributions to the company. Be confident, but maintain a positive and professional tone throughout.\nIt\u0026rsquo;s generally advisable to start your negotiation with a figure slightly higher than your target, giving you room to compromise. However, ensure your request is realistic and based on your research. If the company can\u0026rsquo;t meet your salary requirements, be open to discussing other forms of compensation or benefits.\nDon\u0026rsquo;t feel pressured to accept an offer immediately. It\u0026rsquo;s perfectly acceptable to ask for a few days to review the details. Use this time to carefully consider all aspects of the offer and prepare for further negotiation if necessary.\nFrom personal experience, I can tell you that negotiation is a normal and expected part of the hiring process. In my transition from software engineering to data science, I initially received an offer that was below my expectations. By discussing my skills and the value I could bring to the role, we were able to reach a mutually beneficial agreement that included not just a higher salary, but also additional training opportunities.\nRemember, if you don\u0026rsquo;t ask, you don\u0026rsquo;t get it. Most employers expect candidates to negotiate, and it\u0026rsquo;s often viewed as a positive trait. By approaching the negotiation thoughtfully and professionally, you can ensure that you\u0026rsquo;re fairly compensated for your skills and experience in your new data science role.\nOnce you\u0026rsquo;ve reached an agreement, make sure all the details are documented in your offer letter or contract. This protects both you and the employer and ensures there are no misunderstandings down the line.\nBy following these steps and approaching negotiations with confidence and preparation, you\u0026rsquo;ll be well-positioned to secure a compensation package that reflects your true value as a data scientist.\nConclusion#Starting a career in data science is an exciting journey filled with opportunities for growth, innovation, and impact. As we\u0026rsquo;ve explored throughout this article, landing your dream data science job requires a strategic approach and persistent effort.\nAs you start your data science career, keep in mind that this field is characterised by constant evolution. The learning doesn\u0026rsquo;t stop once you land a job. Stay curious, keep updating your skills, and remain open to new challenges and opportunities.\nYour journey in data science is uniquely yours. There will be challenges along the way, but with perseverance, continuous learning, and a passion for solving complex problems, you can build a rewarding career in this dynamic field.\n","permalink":"https://romeepanchal.com/posts/how_get_data_job/","summary":"A field-tested roadmap for building skills, proof of work, and relationships that lead to a data science role.","title":"A Step-by-Step Guide to Securing a Data Science Job"},{"content":"Today, the internet is flooded with information. It isn\u0026rsquo;t easy to keep up with everything happening in the field. However, to be successful you have to keep yourself informed. The problem is you can only read some things due to time constraints. So, it becomes crucial that whatever little time you spend in consuming information you make the best out of it. But how can you do that? How can you understand and retain most of the information that you read? There’s a simple yet powerful technique that can help you gauge and improve your understanding: summarising content in your own words.\nSummarising is the process of distilling information down to its core ideas and concepts. By writing a summary of what you’ve read without looking back at the original material, you force yourself to engage with the content actively. This process not only reveals how much you’ve understood but also reinforces learning and retention.\nIn this article, we’ll explore the benefits of summarising, break down the process, and provide tips for effectively using this technique to enhance your comprehension and mastery of any subject.\nWhy Summarise the Content#We consume educational content through various mediums. Some prefer watching videos. Others prefer to read. Irrespective of the medium, learning aims to deeply understand the concept.\nBut how do you know if you have thoroughly understood the concept? When you go through the content, you feel fairly confident. However, as time passes you start to forget a lot of details without even realising this fact even if you remember all the details. What if someone asks you a question related to the topic? You will not be able to answer the question unless you have a deep understanding of the topic. At times like this, you realise that there might be a gap in your understanding.\nYou can avoid this pain if you assess your understanding of the topic right after finishing it. There are several techniques to do this. My favourite is summarisation. It is a quick and effective way to assess your understanding of the topic. After going through a topic, take some time and write a summary of it. Writing a summary is not an easy process unless you understand the topic. It requires you to identify the core ideas and concepts. If you struggle to write a summary this would indicate that there is a knowledge gap. The summarisation process will reveal exactly where are you struggling and what you need to work on.\nSummarisation is not only helpful in revealing a knowledge gap. It also improves memory and critical thinking. When you summarise something, it forces your brain to recall the right pieces of information and connect those pieces into a much shorter summary than the original text. It improves your overall comprehension and helps you save a ton of time later.\nHaving understood the importance of summarisation, let’s dive into the practical aspects of how to effectively summarise content.\nThe Summarisation Process#The summarisation process is very straightforward. Here’s how I do it:\nI first identify the core idea of the topic. Then, I identify the supporting ideas and key characteristics of the core idea. Once I’ve identified these details, I write a summary in simple terms, omitting specific details. If I encounter multiple smaller core ideas, I try to connect them and explain how they fit together to form the bigger picture. This reveals a lot about my understanding of the text.\nWhile writing the summary I keep the following things in my mind.\nDid I understand the concept well?Am I able to recognise the characteristics of the idea?Can I make the connection to the other related ideas?Can I write an even shorter summary of this?There are many different techniques to summarise content.\nMind Mapping: Create a visual diagram with the main topic in the centre and related ideas branching out. Start with the central concept, then add major themes as primary branches. Add details and sub-ideas as secondary branches. A mind map helps visualise connections between ideas and organise information hierarchically.The Feynman Technique: No summarisation method is complete without mentioning the famous Feynman technique. Named after physicist Richard Feynman, it involves explaining a concept in simple terms. After reading a topic, try to explain it to a child or someone unfamiliar with the topic. Identify gaps in your explanation and revisit the material. Simplify and use analogies to make the concept clearer. It helps identify gaps in understanding and reinforces learning through teaching.Concept Mapping: Similar to mind mapping, but focuses more on showing relationships between concepts. Write key concepts in boxes or circles, then connect them with labelled lines showing their relationships. Helps understand complex systems and how different ideas relate to each other.Cornell Method: Divide your page into three sections: notes, cues, and summary. Take notes in the largest section during reading or lecture. Write cues/questions in the left column to test your understanding later. Summarise the main ideas in the bottom section. Encourages active engagement with the material and provides a built-in review system.Outlining: Organise information in a hierarchical structure with main points and supporting details. Identify main topics as Roman numerals, subtopics as capital letters, and details as numbers or lowercase letters. It helps structure information logically and see relationships between ideas.How do I use summarisation?#When I first learned linear algebra, I encountered the concepts of vectors and matrices. These are abstract concepts. I learned the formal definitions of vectors and their properties. Then, I learned how they relate to matrices. To solidify my understanding of these abstract concepts, I found it helpful to write summaries of what I learned.\nFor example, I wrote the following summary: Vectors are simply arrays of numbers. There are defined rules for operating on these arrays, such as adding two vectors or multiplying a vector by a number.\nHow do you add two vectors? The first thing to consider is that the vectors you want to add must have the same size. If the first array has 10 numbers, then the second array must also have 10 numbers. To add these arrays, suppose you have two rows. Write one array in the first row and the second array in the second row. Then, look at the corresponding numbers within those arrays and add them together like regular numbers. The result is another array with the same number of elements as the original two arrays.\nSimilarly, to interpret the equation Ax = b, where A is a matrix, x is a vector of unknowns, and b is a resultant vector, I thought of A as a map. Upon multiplication, a matrix transforms vector x to vector b.\nThese kinds of summaries helped me develop the intuition needed to understand the subject. While summarisation is a powerful tool, it’s not without its difficulties. Let’s examine some common challenges and how to overcome them.\nChallenges in Summarisation#While summarisation is a powerful tool for enhancing understanding. However, it comes with its own set of challenges. Recognising these difficulties and learning how to overcome them is crucial for effective summarisation. Let’s explore some common challenges and strategies to address them.\nIt is important to distinguish the main idea from the supporting ideas. Many readers struggle to identify the core concept in a sea of information. Practice the “so what?” test. For each point, ask yourself, “So what? Why is this important?” If it’s crucial to the overall message, it’s likely a main idea. For example, if you are reading about the vectors in linear algebra. You can find the text talks about a bunch of stuff. Ask yourself, so what? The answer should you the importance of vectors.\nAnother important thing to keep in mind is the personal biases. Personal biases can influence how you interpret and summarise information. Try to go through the material with an open mind. Ask as many questions as possible to understand the underlying concept. This will help you overcome your own biases and understand the content better. Then write a summary based on your understanding. Focus on summarising what the text says, not your opinion of it.\nSummaries can become too long or too brief, missing the balance between conciseness and comprehensiveness. Follow the 80/20 rule. Aim to capture 80% of the main content in 20% of the original length.\nConclusion#Throughout this essay, we’ve explored the powerful impact of summarisation on learning and understanding. The key benefits we’ve discussed include:\nEnhanced comprehension of complex topics Improved retention of information Development of critical thinking skills Identification of knowledge gaps More efficient learning and studying Summarisation serves as a versatile tool, applicable across various fields and learning contexts. It not only helps in academic settings but also in professional development and personal growth.\nNow, it’s time to put this knowledge into action. Challenge yourself to incorporate summarisation into your daily learning routine. Start small — perhaps summarise this article you are reading. As you practice, you’ll find your ability to distil and retain information improving significantly.\nIt’s a skill that, once mastered, will serve you well in all areas of life where learning and information processing are key. So, pick up that pen, open that notebook, and start summarising.\n","permalink":"https://romeepanchal.com/posts/general/learning_through_summarisation/","summary":"Why writing summaries exposes gaps in understanding and turns passive reading into durable learning.","title":"Beyond Memorisation: How Summaries Build Deeper Understanding"},{"content":"The bias-variance trade-off is a fundamental concept in machine learning that refers to the trade-off between the ability of a model to fit the training data well (low bias) and its ability to generalize to new, unseen data (low variance). In other words, the bias-variance trade-off is the balance between the complexity and flexibility of a model, and its ability to make accurate predictions on both the training and testing datasets.\nUnderstanding the bias-variance trade-off is critical in machine learning because it helps us to build models that can make accurate predictions on new data. A model with high bias or high variance will not perform well on new data and may lead to incorrect predictions and poor decision-making. Therefore, finding the optimal balance between bias and variance is essential in building robust and accurate machine learning models.\nBias and Variance#In the context of machine learning, bias and variance are two types of errors that can occur in a model.\nBias refers to the difference between the expected or true output and the predicted output of the model. A model with high bias tends to underfit the data, meaning it oversimplifies the relationship between the input features and the target variable. This can result in poor performance on both the training and testing datasets, as the model may not capture all of the relevant patterns in the data.\nOn the other hand, variance refers to the variability of the model\u0026rsquo;s predictions for different training datasets. A model with high variance tends to overfit the data, meaning it captures the noise and idiosyncrasies of the training dataset too well. This can result in poor performance on the testing dataset, as the model may not generalize well to new data.\nLet\u0026rsquo;s consider the example of a regression problem, where we want to predict the price of a house based on its square footage. Here are some examples of models with high and low bias/variance:\nHigh bias, low variance: A linear regression model with only one feature, square footage, would have high bias and low variance. It is too simple to capture the underlying relationship between the square footage and the price of a house, resulting in underfitting. This model would have a high mean squared error on both the training and testing datasets.\nLow bias, high variance: A complex model, such as a neural network with multiple hidden layers, would have low bias and high variance. It has the flexibility to capture the complex, nonlinear relationship between the square footage and the price of a house, but it may overfit the data by capturing the noise and idiosyncrasies of the training dataset too well. This model would have a low mean squared error on the training dataset, but a high mean squared error on the testing dataset.\nOptimal bias-variance trade-off: A model with an optimal balance between bias and variance would have a moderate number of features or hidden layers. For example, a polynomial regression model with second-degree features could capture the underlying nonlinear relationship between the square footage and the price of a house without overfitting. This model would have a low mean squared error on both the training and testing datasets.\nThe Trade-off#How bias and variance are related?#Bias and variance are related in a way that increasing one may decrease the other, and vice versa. This relationship is called the bias-variance trade-off.\nIn machine learning, the bias-variance trade-off refers to the trade-off between the complexity of a model and its ability to generalize well to new data. A model with high bias has low complexity and may be too simple to capture the underlying patterns in the data. This model is likely to underfit the data and has poor performance on both the training and testing datasets. On the other hand, a model with high variance has high complexity and may be too flexible, resulting in overfitting and poor performance on the testing dataset.\nTo achieve a good trade-off between bias and variance, we need to find the optimal complexity of the model. This means that we need to increase the complexity of the model until it captures the relevant patterns in the data, but not so much that it captures the noise and idiosyncrasies of the training dataset. The optimal complexity is the one that minimizes the total error, which is the sum of the bias and variance.\nIn practice, we can control the bias-variance trade-off by adjusting the hyperparameters of the model, such as the number of features or the regularization strength. For example, we can increase the complexity of a model by adding more features or using a more powerful algorithm, which reduces bias but increases variance. Conversely, we can decrease the complexity of a model by using regularization techniques, which increases bias but reduces variance.\nHow increasing/decreasing one affects the other?#Increasing one bias or variance in a model can affect the other in several ways.\nIf we increase the bias in a model, we decrease its complexity and make it simpler. This means that the model is more likely to underfit the data and may not be able to capture the underlying patterns. As a result, the training error and testing errors both increase. However, increasing the bias can reduce the variance of the model because it is less sensitive to the noise and idiosyncrasies of the training data. So, while increasing the bias may reduce overfitting, it can also make the model less flexible and less able to capture complex patterns.\nConversely, if we increase the variance in a model, we increase its complexity and make it more flexible. This means that the model is more likely to overfit the data and may capture noise or idiosyncrasies in the training data that are not present in the testing data. As a result, the training error may decrease while the testing error increases, indicating poor generalization performance. However, increasing the variance can reduce bias because the model is better able to capture the underlying patterns in the data. So, while increasing the variance may improve the model\u0026rsquo;s flexibility, it can also make it more prone to overfitting and poor generalization performance.\nTherefore, increasing one bias or variance in a model can lead to a trade-off between the two. To achieve optimal performance, we need to find the right balance between bias and variance that minimizes the total error of the model. This is why the bias-variance trade-off is an important concept in machine learning, as it helps us to choose the best model complexity for a given problem.\nUnderfitting and Overfitting#Underfitting and overfitting are two common problems that can arise in machine learning when trying to build models that can generalize well to new data.\nUnderfitting occurs when the model is too simple and is not able to capture the underlying patterns in the data. As a result, the model may perform poorly on both the training and testing data, indicating that it has not learned the relevant features of the data. This can happen when the model has high bias and low variance, and it is not able to capture the complexity of the data.\nOverfitting occurs when the model is too complex and is too tightly fit to the training data. As a result, the model may perform very well on the training data but poorly on the testing data, indicating that it has learned noise or idiosyncrasies in the training data. This can happen when the model has low bias and high variance, and it is too flexible and overfits the training data.\nUnderfitting and overfitting are related to the bias-variance trade-off because they represent two extremes of the trade-off. Underfitting is typically associated with high bias and low variance while overfitting is typically associated with low bias and high variance. In other words, underfitting occurs when the model is too simple and does not capture the complexity of the data, leading to high bias and low variance. On the other hand, overfitting occurs when the model is too complex and captures the noise and idiosyncrasies of the training data, leading to low bias and high variance.\nExamples of underfitting and overfitting are as follows:\nUnderfitting: A linear regression model that tries to fit a non-linear relationship between the features and the target variable. The model is too simple and cannot capture the non-linear patterns in the data, leading to high bias and low variance.Overfitting: A decision tree model with very deep and complex trees that capture every detail of the training data. The model is too flexible and captures the noise and idiosyncrasies of the training data, leading to low bias and high variance.To avoid underfitting and overfitting, we need to find the optimal balance between bias and variance that minimizes the total error of the model. This requires choosing the appropriate model complexity, such as adjusting the hyperparameters, using regularization techniques, or trying different algorithms. By finding the right balance between bias and variance, we can build models that generalize well to new data and achieve optimal performance.\nFinding the Optimal Model Complexity#Finding the optimal model complexity involves balancing the bias-variance trade-off to minimize the total error of the model. This requires choosing the appropriate model complexity that can capture the relevant features of the data without overfitting or underfitting. Several techniques can be used to find the optimal model complexity, including cross-validation, regularization, and model selection.\nCross-validation is a technique used to evaluate the performance of a model by splitting the data into training and testing sets, and iteratively training the model on different subsets of the data. This allows us to estimate the generalization error of the model and choose the appropriate complexity that minimizes the error. For example, k-fold cross-validation involves splitting the data into k subsets and iteratively training the model on k-1 subsets and evaluating the performance on the remaining subset. This is repeated k times with different subsets used for testing each time.\nRegularization is a technique used to prevent overfitting by adding a penalty term to the objective function of the model. This penalty term discourages the model from fitting the noise or idiosyncrasies in the data and encourages it to focus on the relevant features. Examples of regularization techniques include L1 and L2 regularization, which add a penalty term based on the magnitude of the weights of the model, and dropout, which randomly drops out some of the neurons in a neural network during training to prevent overfitting.\nModel selection is a technique used to choose the appropriate algorithm and hyperparameters for the model. This involves trying different algorithms with different hyperparameters and evaluating their performance using cross-validation or other techniques. Examples of model selection techniques include grid search, which exhaustively searches the hyperparameter space for the best combination of hyperparameters, and random search, which randomly samples the hyperparameter space to find a good combination of hyperparameters.\nFor example, suppose we are building a classification model to predict whether a customer will churn or not based on their demographic and behavioural data. We start by trying different algorithms, such as logistic regression, decision trees, and neural networks, with different hyperparameters. We use k-fold cross-validation to evaluate the performance of each algorithm and choose the best one based on its generalization error. Once we have chosen the best algorithm, we use regularization techniques such as L1 or L2 regularization to prevent overfitting and fine-tune the hyperparameters using grid search or random search. This allows us to find the optimal model complexity that balances bias and variance and achieves the best performance on the testing data.\nReal-World Applications#The bias-variance trade-off is a fundamental concept in machine learning that applies to a wide range of real-world scenarios. Here are some examples of how the trade-off applies in real-world scenarios:\nMedical diagnosis: In medical diagnosis, the bias-variance trade-off can impact the accuracy of the diagnosis. A high-bias model may oversimplify the diagnosis, leading to a high error rate due to underfitting. On the other hand, a high-variance model may overfit the diagnosis, leading to low accuracy due to noise in the data. A model with an appropriate bias-variance balance can achieve high accuracy and reliable diagnosis.\nFinancial forecasting: In financial forecasting, the bias-variance trade-off can affect the interpretability of the model. A high-bias model may produce simple and interpretable forecasts but may fail to capture complex patterns and underlying relationships in the data. A high-variance model may produce accurate forecasts but may be difficult to interpret and explain due to its complexity.\nImage recognition: In image recognition, the bias-variance trade-off can impact the robustness of the model. A high-bias model may miss subtle patterns in the image, leading to low accuracy and poor recognition performance. A high-variance model may fit the noise in the data, leading to overfitting and poor generalization performance. A model with an appropriate bias-variance balance can achieve high accuracy and robust recognition performance.\nThe bias-variance trade-off affects the accuracy and interpretability of models in different ways. A model with high bias tends to be simpler and more interpretable but may sacrifice accuracy due to underfitting. A model with high variance tends to be more complex and less interpretable but may achieve high accuracy on the training data due to overfitting. The optimal balance between bias and variance depends on the specific problem and dataset. In general, a model with moderate complexity and an appropriate bias-variance balance can achieve high accuracy and interpretability.\nConclusion#Understanding the bias-variance trade-off is essential in machine learning because it allows us to develop models that generalize well to unseen data. By finding the optimal balance between bias and variance, we can create models that are both accurate and interpretable, making them useful in real-world applications.\nIn conclusion, the bias-variance trade-off is a critical concept for machine learning practitioners to understand. By balancing the trade-off between bias and variance, we can develop models that are both accurate and interpretable, resulting in better performance and better results in real-world applications.\n","permalink":"https://romeepanchal.com/posts/data_science/bias_variance_trade-off/","summary":"Understand underfitting, overfitting, and why every useful model must balance bias against variance.","title":"Bias Variance Trade-off"},{"content":"In today\u0026rsquo;s data-driven world, businesses hold a treasure trove of information. But simply having data isn\u0026rsquo;t enough. The true power lies in extracting valuable insights to inform better decision-making and fuel growth. This article will guide you through a strategic approach to uncover hidden gems within your data.\nWhat are Insights?#Insights are not just random observations; they are previously unknown, actionable business trends or knowledge. They provide a deeper understanding of your customers, market dynamics, or internal operations, empowering you to take strategic actions.\nAsking Questions#Before diving headfirst into data exploration, it\u0026rsquo;s crucial to have a clear direction. Start by formulating insightful questions that align with your business goals. Here\u0026rsquo;s why this is important:\nFocused Exploration: Asking specific questions prevents you from getting lost in a maze of data. It ensures your analysis is targeted and delivers valuable results.\nUncovering the Unknown: By asking the right questions, you may unearth patterns or relationships that were previously hidden. This can lead to surprising discoveries that can significantly impact your business.\nExamples of Insightful Questions#Imagine you\u0026rsquo;re analyzing sales data. Here are some questions to get you started:\nWhat are the sales trends over the past year? Are there any seasonal fluctuations?Who are our most valuable customers? What are their key characteristics?Do specific regions or outlets consistently outperform others?Is there a correlation between marketing campaigns and sales spikes?Beyond the Obvious#While some initial questions might reveal readily apparent trends, the true gems lie deeper. Don\u0026rsquo;t be discouraged if your initial findings seem familiar. Here\u0026rsquo;s how to push the exploration further:\nBusiness Context: Gain a deeper understanding of your industry and how your company operates. Talk to colleagues from different departments to gain a well-rounded perspective.\nHypothesis-Driven Analysis: Based on your understanding, formulate hypotheses about the data. For instance, you might hypothesize that sales data follows a normal distribution.\nData Validation: Use statistical methods or visualizations to test your hypotheses. Deviations from your expectations can be significant insights.\nFinally Something Interesting#If the data doesn\u0026rsquo;t conform to your expectations, congratulations! You\u0026rsquo;ve potentially stumbled upon a valuable insight. Investigate further to understand the reasons behind the anomaly. This deeper exploration can lead to game-changing discoveries.\nConclusion#By combining a question-driven approach with a solid understanding of your business, you can transform raw data into actionable insights. Remember, data is a powerful tool, but it\u0026rsquo;s the questions you ask that unlock its true potential. So, start asking, explore, and unearth the hidden gems waiting to be discovered within your data!\n","permalink":"https://romeepanchal.com/posts/how_find_insights/","summary":"A practical framework for moving beyond charts and finding observations that change decisions.","title":"How to Find Insights in Data"},{"content":"I am Romee Panchal, an AI engineer with more than nine years of experience across software engineering and data science.\nI am drawn to difficult technical problems and to the equally difficult work of explaining them clearly. This blog is where I connect those interests: unpacking machine learning systems, tracing how ideas evolve, and turning practical experience into useful mental models.\nI write about AI, machine learning, and engineering for practitioners who want to understand not just what works, but why. Most of the essays are attempts to make intelligent systems feel less magical and more inspectable.\nYou will find essays about machine learning, natural language processing, deep learning, data science, and the craft of learning technical subjects well. My aim is not to repeat documentation. It is to explain why an approach works, where it breaks, and how to think about the trade-offs.\nOutside the blog, you can find me on X, LinkedIn, and GitHub.\n↳ Follow the work\nGet new essays about AI and machine learning through my Substack newsletter. ","permalink":"https://romeepanchal.com/about/","summary":"AI engineer, problem solver, and technical writer.","title":"About Romee"}]