[{"content":"","date":"4 September 2025","externalUrl":null,"permalink":"/","section":"Home","summary":"","title":"Home","type":"page"},{"content":"","date":"4 September 2025","externalUrl":null,"permalink":"/posts/","section":"Posts","summary":"","title":"Posts","type":"posts"},{"content":" You can experience the power of quantized, compressed embeddings for yourself—right here in the browser. The demo below loads a compact 1.3MB MessagePack file containing both model weights and song data. Thanks to browser caching, repeat visits are lightning-fast.\nFor the full technical walkthrough and strategic insights on quantized and compressed embeddings, see my earlier post. Give it a spin and see real-time, serverless inference in action!\nSong Identifier Get Recommendations ","date":"4 September 2025","externalUrl":null,"permalink":"/posts/2025/demo-edge-song-recommendation/","section":"Posts","summary":"","title":"Try It Live: Edge-Based Song Recommendation","type":"posts"},{"content":" One of the most fascinating challenges in deploying machine learning models isn\u0026rsquo;t the algorithm itself—it\u0026rsquo;s how to serve them efficiently to real users at scale.\nWhile large language models often grab the spotlight, there\u0026rsquo;s tremendous value in lightweight, task‑specific embedding models—especially when you want real‑time inference directly in the browser. In this post, I\u0026rsquo;ll walk through how I took a Word2Vec embedding trained on playlist data and prepared it for fast, compressed inference at the edge using quantization techniques and WebAssembly deployment with Rust.\nWith consulting in mind, I\u0026rsquo;ll also highlight the broader implications for strategy, cost, and user experience—because for businesses, the story isn\u0026rsquo;t just about vectors and compression, but about time‑to‑value, scalability, and user adoption.\nThe Problem: Delivering Embeddings at the Edge # The dataset:\n75,262 songs (vocabulary)\n11,088 playlists used for training\nLearned embedding: 75,261 × 32 matrix (~9MB) in float32\nThe inference task is simple: given a song, find the most similar songs by cosine similarity, and generate a playlist.\nOn a local machine, this is computationally trivial. But deploying to a static browser‑based application raises a bottleneck: how do we deliver the model weights efficiently over the network?\nFor context:\nThe trained Word2Vec model in gensim weighs ~21MB (because it stores additional metadata for training continuation). Just the weight matrix in binary comes to ~9.7MB. This might be acceptable for cloud inference, but for browser inference, I wanted a sub‑MB solution to minimize time‑to‑first‑recommendation and preserve user experience. This is where quantization and compression enter the picture.\nQuantization: Shrinking Float32 to UInt8 # Quantization is an increasingly popular method across the ML landscape. At the core, we map a continuous float32 range down to very compact integers (uint8), with a scale and zero point for reconstruction.\nThis reduces file size by ~4× while still preserving the rank order of similarities.\nAfter applying min‑max quantization:\nOriginal Float32 weights: 9.7MB Quantized UInt8 weights: 2.3MB Decompress error analysis: MSE on embedding: \\(\\sim 0.0012\\) MSE on L2 norms: \\(\\sim 0.0027\\) Top‑k similarity results: ranking only slightly perturbed (nearly invisible to a casual playlist user). ➡️ Strategic takeaway: With minimal loss in quality, we achieve ~4× smaller load times, making real‑time edge inference possible even in constrained environments.\nCompression: From MBs to KBs # Modern browsers support gzip and brotli decompression natively. By precompressing my quantized weights:\nGzip → 779KB\nBrotli (max level) → 609KB\nNow we\u0026rsquo;re talking about sub‑second load times on broadband, making the experience seamless.\n➡️ Consulting angle: Aggressively tuned compression means faster customer engagement on first use, lower infrastructure costs, and enables deployment to bandwidth-sensitive markets—a major strategic advantage.\nRust + WebAssembly: Running It in the Browser # With the optimized model file ready, I then built the inference engine in Rust, compiled it to WebAssembly, and used wasm-bindgen for browser interop.\nWhy Rust + WASM?\nPerformance: Tight loops for matrix multiplication and top‑k search execute at near‑native speed.\nSafety: No runtime crashes, predictable memory model.\nPortability: Runs uniformly across all major browsers without backend servers.\n➡️ Business impact: Zero server requirements. Inference literally happens in the user\u0026rsquo;s browser. That means no cloud hosting cost, simplified data governance (no user data leaves the device), and instant scaling from 10 users to 10 million users at no incremental compute cost.\nStrategic Implications Beyond Playlists # While this experiment focused on song recommendation, the same approach scales to a variety of business contexts:\nRetail: Lightweight on‑device recommendation systems for cross‑selling.\nFinance: Privacy‑preserving edge scoring for customer qualification.\nHealthcare: Embedding‑based retrieval for medical knowledge, fully offline.\nEmerging markets: Model delivery for low‑bandwidth regions, improving inclusivity.\nWhat excites me here is the intersection of technical optimization and business value creation. This is the type of \u0026ldquo;edge strategy\u0026rdquo; that allows companies to ship smarter products faster, while being mindful of cost, trust, and accessibility.\nWrapping Up # From a 21MB training artifact to a 609KB ready‑to‑deploy package, this work shows that quantization and compression aren\u0026rsquo;t just academic tricks—they\u0026rsquo;re enablers of viable product strategy. Pulling this together in Rust + WebAssembly demonstrates that with the right technical toolset, even complex ML inference can move to the browser with confidence.\nFor me, the broader takeaway is this:\nSmart compression unlocks strategic edge deployment.\nStay tuned—I\u0026rsquo;ll be releasing a demo of the full pipeline so that you can experience quantized song embedding live in the browser.\n","date":"28 August 2025","externalUrl":null,"permalink":"/posts/quantized-compressed-embeddings-edge-inference/","section":"Posts","summary":"","title":"Quantized and Compressed Embeddings for Fast Edge Inference with Rust and WebAssembly","type":"posts"},{"content":"","date":"17 August 2024","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"17 August 2024","externalUrl":null,"permalink":"/categories/prescriptive-analytics/","section":"Categories","summary":"","title":"Prescriptive Analytics","type":"categories"},{"content":" In the competitive landscape of modern business, minimizing operational costs is essential for maintaining profitability. Transportation costs, in particular, play a significant role in the overall cost structure of companies that rely on distributing goods from manufacturing plants to various markets. Efficiently optimizing these transportation routes can lead to significant savings and improved service delivery.\nIn this blog post, we\u0026rsquo;ll explore how Python can be used to solve transportation problems, focusing on minimizing transportation costs when distributing goods from plants to markets. We\u0026rsquo;ll walk through the problem, define the necessary variables and constraints, and demonstrate how Python can be used to implement an optimal solution.\nUnderstanding the Problem # The transportation problem is a classic optimization challenge in operations research. It involves determining the most cost-effective way to transport goods from a set of suppliers (e.g., plants) to a set of consumers (e.g., markets). The objective is to minimize the total transportation cost while satisfying supply and demand constraints.\nInput Parameters # To model the problem, we need three key input parameters:\nSupply: The available quantity of goods at each plant. Demand: The required quantity of goods at each market. Transportation Costs: The cost of transporting one unit of goods from a particular plant to a particular market. For those interested in the mathematical derivation, jump right in to the math.\nMath Magician's Secret Scroll The transportation problem can be expressed as a linear programming model. Let \\(x_{ij}\\) represent the number of units transported from plant \\(i\\) to market \\(j\\). The objective is to minimize the total cost:\ngraph TD A[Plant i] --\u003e|x_ij| B[Market j] $$ \\text{Minimize } Z= \\sum_{i=1}^m\\sum_{j=1}^nc_{ij}x_{ij} $$Subject to:\nSupply constraints: \\(\\sum_{j=1}^nx_{ij}\\leq s_i\\) for all plants \\(i \\in \\{1,..,m\\}\\) Demand constraints: \\(\\sum_{i=1}^mx_{ij}\\leq d_i\\) for all markets \\(j \\in \\{1,..,n\\}\\) Non-negativity: \\(x_{ij} \\geq 0\\) Where:\n\\(c_{ij}\\) is the cost of transporting one unit from plant \\(i\\) to market \\(j\\) \\(s_i\\) is the supply at plant \\(i\\) \\(d_j\\) is the demand at market \\(j\\) The solution to this linear programming problem yields the optimal transportation plan and minimum cost.\nIf you\u0026rsquo;d rather see the optimization in action, follow through.\nimport sympy as sp from scipy.optimize import linprog We have all the required packages loaded and ready. Let us now implement the input parameters in python.\n# Step 1: Define the supply at each plant supply = [20, 30, 25] plant_names = [\u0026#39;Plant 1\u0026#39;, \u0026#39;Plant 2\u0026#39;, \u0026#39;Plant 3\u0026#39;] # Step 2: Define the demand at each market demand = [15, 25, 35] market_names = [\u0026#39;Market A\u0026#39;, \u0026#39;Market B\u0026#39;, \u0026#39;Market C\u0026#39;] # Step 3: Define the transportation costs from each plant to each market transportation_costs = [ [8, 9, 14], # Costs from Plant 1 to Market A, B, C [6, 12, 9], # Costs from Plant 2 to Market A, B, C [10, 13, 16] # Costs from Plant 3 to Market A, B, C ] print(\u0026#34;Input parameters defined!\u0026#34;) Decision Variables # In optimization problems, decision variables represent the quantities we want to determine. In this case, the decision variables will represent the number of units to be transported from each plant to each market.\nNote: The decision variables are the unknowns that we need to solve for. These variables are subject to constraints (supply and demand) and are used in the objective function to calculate the total transportation cost.\n# Step 4: Create symbolic variables for the units transported X = sp.Matrix([[sp.symbols(f\u0026#39;X_{i+1}_{j+1}\u0026#39;) for j in range(len(market_names))] for i in range(len(plant_names))]) sp.pprint(X) Resource Constraints # Resource constraints ensure that the supply from each plant is not exceeded and the demand at each market is met.\nSupply Constraints # Each plant can only supply a limited number of goods, so the sum of goods transported from a plant to all markets cannot exceed the plant\u0026rsquo;s supply.\n# Step 5: Define the supply constraints symbolically supply_constraints = [] for i in range(len(plant_names)): constraint = sp.Eq(sum(X[i, j] for j in range(len(market_names))), supply[i]) supply_constraints.append(constraint) print(\u0026#34;Supply Constraints:\u0026#34;) for constraint in supply_constraints: sp.pprint(constraint) Demand Constraints # Each market requires a certain amount of goods, so the sum of goods received from all plants should meet the market\u0026rsquo;s demand.\n# Step 6: Define the demand constraints symbolically demand_constraints = [] for j in range(len(market_names)): constraint = sp.Eq(sum(X[i, j] for i in range(len(plant_names))), demand[j]) demand_constraints.append(constraint) print(\u0026#34;Demand Constraints:\u0026#34;) for constraint in demand_constraints: sp.pprint(constraint) Objective Function # The objective is to minimize the total transportation cost, which is the sum of the cost for each route multiplied by the number of units transported.\n# Step 7: Define the objective function symbolically (minimize transportation costs) objective_function = sum(X[i, j] * transportation_costs[i][j] for i in range(len(plant_names)) for j in range(len(market_names))) # Display the objective function and constraints print(\u0026#34;Objective Function:\u0026#34;) sp.pprint(objective_function) Solving the Problem with Python # With the decision variables, constraints, and objective function defined, we can now solve the transportation problem using Python. Since the API wise intuitive pulp package does not work in browser yet, let us use Scipy\u0026rsquo;s solver. However, it requires us to convert the problem to a form that the API understands. Let us do that first.\n# Step 8: Convert the problem to a form that can be solved by linprog # Flatten the transportation costs and convert the objective function to a linear cost vector c = [transportation_costs[i][j] for i in range(len(plant_names)) for j in range(len(market_names))] # Create the equality constraint matrix for supply (Ax = b) A_eq = [] b_eq = supply + demand # Supply constraints (each row ensures the sum of shipments from a plant equals the plant\u0026#39;s supply) for i in range(len(plant_names)): row = [1 if k // len(market_names) == i else 0 for k in range(len(plant_names) * len(market_names))] A_eq.append(row) # Demand constraints (each row ensures the sum of shipments to a market equals the market\u0026#39;s demand) for j in range(len(market_names)): row = [1 if k % len(market_names) == j else 0 for k in range(len(plant_names) * len(market_names))] A_eq.append(row) # Convert to the appropriate format for linprog A_eq = sp.Matrix(A_eq).tolist() b_eq = sp.Matrix(b_eq).tolist() # Step 9: Define the bounds for decision variables (non-negative) bounds = [(0, None) for _ in range(len(c))] print(\u0026#34;Ready for optimization!\u0026#34;) Warning: The quantity transported from each plant to each market is always non-negative.\nNow, we can go ahead and solve the cost minimization problem.\n# Step 10: Solve the linear programming problem using scipy.optimize.linprog result = linprog(c, A_eq=A_eq, b_eq=b_eq, bounds=bounds, method=\u0026#39;highs\u0026#39;) # Extract and print the results optimal_values = result.x.reshape((len(plant_names), len(market_names))) print(\u0026#34;Optimal Transportation Matrix:\u0026#34;) print(\u0026#34; \u0026#34; * 12, end=\u0026#34;\u0026#34;) # Print space for the column headers for market in market_names: print(f\u0026#34;{market:\u0026gt;10}\u0026#34;, end=\u0026#34;\u0026#34;) # Print market names as column headers print() # New line after column headers for i, plant in enumerate(plant_names): print(f\u0026#34;{plant:\u0026gt;10}\u0026#34;, end=\u0026#34;\u0026#34;) # Print plant names as row headers for j in range(len(market_names)): print(f\u0026#34;{optimal_values[i, j]:10.2f}\u0026#34;, end=\u0026#34;\u0026#34;) # Print the cost values print() # New line after each row print(\u0026#34;Detailed Optimal Transportation Plan:\u0026#34;) for i in range(len(plant_names)): for j in range(len(market_names)): print(f\u0026#34;Units transported from {plant_names[i]} to {market_names[j]}: {optimal_values[i, j]:.2f}\u0026#34;) print(f\u0026#34;\\nTotal Transportation Cost: ${result.fun:.2f}\u0026#34;) Conclusion # Optimizing transportation costs is crucial for businesses looking to improve their efficiency and bottom line. By formulating the transportation problem as a linear programming model and solving it with Python, companies can identify cost-effective strategies for distributing goods from plants to markets.\nPython\u0026rsquo;s flexibility and powerful libraries, such as pulp, make it an excellent choice for tackling optimization problems. The code examples in this post provide a hands-on approach for solving real-world transportation challenges.\n","date":"17 August 2024","externalUrl":null,"permalink":"/posts/business-optimization/01-solving-transportation-problems-with-python/","section":"Posts","summary":"","title":"Solving Transportation Problems with Python: Minimizing Transportation Costs","type":"posts"},{"content":"","date":"17 August 2024","externalUrl":null,"permalink":"/categories/supply-chain-management/","section":"Categories","summary":"","title":"Supply Chain Management","type":"categories"},{"content":" I recently encountered the classical gaps and islands problem, an intriguing issue where the goal is to label clusters of consecutive sequences that contain gaps or missing values. Each cluster, or \u0026ldquo;island,\u0026rdquo; needs a distinct label so that aggregate functions can be applied to each one separately.\nNote: The \u0026ldquo;gaps and islands\u0026rdquo; problem in SQL is a common scenario where you need to identify contiguous sequences (islands) and gaps in a dataset.\nIn this post, we will explore two simple business cases for gaps and islands. It is important to note that there are multiple ways to implement a solution for gaps and islands, including subqueries, the rank function with CTE, and cursors. I will focus on the rank function with CTE as I find it neat and effective.\nInventory Status # Businesses maintaining an inventory often need to track periods when a product is continuously in stock or out of stock. This can help understand supply chain issues and the demand patterns for various products. Such insights can aid businesses in making informed purchasing decisions, thus avoiding overstocking or understocking.\nI have already set up the InventoryStatus table. You may run it to study the inventory status of two products for a six-day period from 25th June to 30th June 2024.\nTip: The SQL blocks are editable!!\nselect * from InventoryStatus; Let\u0026rsquo;s proceed with the rank function with CTE approach. Here, we use two CTEs with the second one referencing the first. Run the query below to see it in action.\nWITH StatusPeriods AS ( SELECT ProductID, StatusDate, InStock, ROW_NUMBER() OVER (PARTITION BY ProductID ORDER BY StatusDate) - ROW_NUMBER() OVER (PARTITION BY ProductID, InStock ORDER BY StatusDate) AS GroupID FROM InventoryStatus ), Islands AS ( SELECT ProductID, InStock, MIN(StatusDate) AS StartPeriod, MAX(StatusDate) AS EndPeriod, JULIANDAY(MAX(StatusDate)) - JULIANDAY(MIN(StatusDate)) AS Days FROM StatusPeriods GROUP BY ProductID, InStock, GroupID ) SELECT * FROM Islands ORDER BY ProductID, StartPeriod; In the StatusPeriods CTE, two row numbers are calculated:\nOne based on the order of StatusDate partitioned by ProductID.\nOne based on the order of StatusDate partitioned by ProductID and InStock.\nThe difference between these row numbers (GroupID) helps identify contiguous periods by grouping rows where the difference is the same.\nThe Islands CTE groups by ProductID, InStock, and GroupID to identify contiguous periods, selecting the minimum StatusDate and maximum StatusDate as the start and end of each period, with Days representing the duration of each contiguous period.\nContinuous Subscription Periods # Subscription models are prevalent across many industries, from music (Spotify) and movie streaming (Netflix) to online shopping (Amazon Prime) and food services (Zomato Gold). Predictable revenue, reducing churn, forecasting inventory, valuable customer data, and increased average order value are just a few of the many motivators for businesses adopting this model. The psychology behind consumer subscription behavior are also quite compelling for this wide adoption.\nI have already set up the Subscriptions table with dummy data for the year 2024. You may run it to examine the data.\nNote: A subscription period is considered continuous if the next subscription starts on the day following the end of the previous subscription.\nselect * from Subscriptions; As before, we will implement the rank function with CTE approach. Run the following query:\nWITH SubscriptionPeriods AS ( SELECT CustomerID, StartDate, EndDate, LAG(EndDate, 1) OVER (PARTITION BY CustomerID ORDER BY StartDate) AS PrevEndDate FROM Subscriptions ), Islands AS ( SELECT CustomerID, StartDate, EndDate, SUM(CASE WHEN PrevEndDate IS NULL OR JULIANDAY(StartDate) - JULIANDAY(PrevEndDate) \u0026gt; 1 THEN 1 ELSE 0 END) OVER (PARTITION BY CustomerID ORDER BY StartDate) AS IslandGroup FROM SubscriptionPeriods ) SELECT CustomerID, MIN(StartDate) AS StartPeriod, MAX(EndDate) AS EndPeriod FROM Islands GROUP BY CustomerID, IslandGroup ORDER BY CustomerID, StartPeriod; In the SubscriptionPeriods CTE, the LAG function is used to get the EndDate of the previous subscription row for each customer.\nIn the Islands CTE, a CASE statement determines if the current row should start a new group. A new group starts if:\nPrevEndDate is NULL (i.e., it\u0026rsquo;s the first row for the customer).\nThe difference in days between PrevEndDate and StartDate is more than one day.\nA running total (SUM with CASE) is calculated to assign a unique IslandGroup identifier for each contiguous subscription period.\nThe final query performs the grouping for islands. This approach can be similarly applied to other business cases, such as calculating the continuous work period for employees or the vacation periods of employees.\nBy understanding and implementing the gaps and islands problem, businesses can gain valuable insights into various operational aspects, enabling more informed decision-making.\n","date":"6 July 2024","externalUrl":null,"permalink":"/posts/db/01-sql-gaps-and-islands/","section":"Posts","summary":"","title":"Mastering SQL Gaps and Islands: Solutions for Inventory and Subscription Period Analysis","type":"posts"},{"content":"On December 6th, 2022, I successfully defended my master\u0026rsquo;s thesis at Indian Institute of Science (IISc). While I had considered pursuing an MBA, I chose to focus on my thesis and didn\u0026rsquo;t take the CAT to avoid overloading myself. During this time, I was also exploring data science through an online course, honing my technical skills in math, programming, and computer science. Despite my technical proficiency, I felt uncertain about navigating the corporate world.\nIn February, my dad informed me about an upcoming MBA program entrance exam at Birla Institute of Technology and Science, Pilani. Although the Business Analytics program was relatively new, I decided to apply. Receiving the offer letter presented a pivotal decision: continue interviewing for jobs and retake the CAT next year, or embrace the MBA opportunity. With strong ROI and my family\u0026rsquo;s support, I saw the MBA as a chance to transition from academia to the corporate realm, retaining my technical edge while enhancing my interpersonal skills. I chose to join the program. Connecting with peers before arriving on campus eased my transition, and a month into the course, I was nominated as a Core Member of the Student Faculty Council by popular demand.\nWhile I found the management courses challenging, I excelled in the technical ones. Unexpectedly, I formed close friendships with two peers who provide honest feedback and encourage my growth. One friend, in particular, emphasized the importance of networking, leading to my off-campus summer internship. Shoutout to Kirti Bhushan Ganguly and Abhinav Kathuria for being incredible friends and mentors. Your support and constructive criticism have been invaluable.\nA professor once told me that an MBA teaches frameworks that structure your thinking and benefit you throughout life. Reflecting on my journey, I wholeheartedly agree. Although I still have much to learn, I am grateful for the experiences and growth over the past year. This journey has been a blend of perseverance, learning, and invaluable personal and professional development.\nIf I had to describe my MBA journey so far in a word, it would be \u0026lsquo;flexibility\u0026rsquo;—adaptability to challenges, openness to new experiences, and resilience in learning.\n","date":"29 June 2024","externalUrl":null,"permalink":"/posts/mba-year-1/","section":"Posts","summary":"","title":"One year of MBA","type":"posts"},{"content":" A time series, represented as \\(x_1, x_2, x_3, \\ldots\\), forms a stochastic process denoted by \\(\\{x_t\\}\\) indexed by time \\(t\\).\nStationarity: A Foundation of Stability # Stationarity in stochastic processes denotes a regularity where statistical properties like mean, variance, and autocovariance remain constant over time. Essentially, a Stationary Stochastic Process signifies that the behavior of the data doesn\u0026rsquo;t alter as time progresses.\ngraph LR A[Stationary Process] --\u003e B[Strictly Stationary Process] A --\u003e C[Weakly Stationary Process] C --\u003e D[Covariance Stationary Process] C --\u003e E[Trend-Stationary Process] C --\u003e F[Seasonal Stationary Process] Strictly Stationary Process # This type maintains identical probabilistic behavior across all time shifts. The joint distribution of any set of time points within the process remains invariant to time shifts, ensuring constancy in statistical properties over time.\n$$ Pr\\{x_{t1}\\leq c_1,\\ldots,x_{tk} \\leq c_k\\} = Pr\\{x_{t1+h}\\leq c_1,\\ldots,x_{tk+h} \\leq c_k\\} $$ Consequence: The moments of the stochastic process also remain constant over time. However, strict stationarity might be too stringent for practical applications. Instead, a milder version, known as a Weakly Stationary Process, imposes conditions only on the first two moments of the series.\nWeakly Stationary Process # This type is characterized by a constant mean value function (\\(μ_t\\)) and an autocovariance function (\\(\\gamma(s, t)\\)) that depends solely on the difference between time points (\\(|s - t|\\)).\nImplication: Regularity in mean and autocorrelation functions facilitates estimations through averaging, ensuring stability in analysis. Moving beyond weak stationarity, specific types of stationary processes delineate variations in mean, variance, and autocovariance:\nCovariance Stationary Process: This variant maintains constant mean and autocovariance over time, allowing for changes in variance that are not systematic or trend-based.\nTrend-Stationary Process: Here, the mean remains constant, but variations in variance or autocovariance are attributed to a deterministic trend in the data. Removal of the trend renders the series stationary.\nSeasonal Stationary Process: Displaying seasonal patterns, this process features consistent mean and variance within each season, although these metrics may vary across seasons. Eliminating the seasonal component transforms the series into a stationary form.\nUnderstanding these distinctions in stationarity is crucial for robust analysis and modeling of time series data. By grasping the stability and regularity inherent in different stationary processes, analysts can derive more accurate insights and predictions from their data.\n","date":"7 May 2024","externalUrl":null,"permalink":"/posts/stationary-stochastic-process/","section":"Posts","summary":"This blog post cuts through the complexity to explain stationary stochastic processes simply and directly. Discover what stationarity means, explore different types like strictly stationary, weakly stationary, and more.","title":"Understanding Stationary Stochastic Processes","type":"posts"},{"content":"","date":"24 November 2022","externalUrl":null,"permalink":"/tags/datascience/","section":"Tags","summary":"","title":"Datascience","type":"tags"},{"content":"I came across the Game of Thrones script for the first time in the final project of a Python course by Internshala. The idea for the project was to simply find unique words spoken by the characters. I, for one, wanted to explore some visualizations, and here is my attempt at one. We will find the character with the maximum number of lines in the script and create a word cloud. Fairly simple stuff.\nWho is the character with the maximum number of lines in the script, and what were the words they spoke the most?\nThe Dataset # I found a GOT dataset in the public domain graciously delivered by Alben Tumanggor in Kaggle. We will be working with this dataset for our explorations. Here is the link if you wanna explore it on your own.\nGoing over the headers for each column and what they correspond to will give us a good start. You can find a summary in the table.\nHeader Description Example Release Date The original air date of the episode in YYYY-MM-DD format. 2011-04-17 Season The season number. Season 1 Episode The episode number. Episode 1 Episode Title The title of the episode. Winter is Coming Name Name of the GOT character. waymar royce Sentence Sentence spoken by the character. What do you expect? They\u0026rsquo;re savages. One lot steals a goat from another lot and before you know it, they\u0026rsquo;re ripping each other to pieces. Let us import the packages that we will use for the analysis.\n%matplotlib inline import pandas as pd import numpy as np import matplotlib.pyplot as plt import re Before we read the whole dataset, I feel more comfortable having a glimpse first. So we will read five rows from the dataset to get started.\ninput_file_path = \u0026#39;../input/game-of-thrones-script-all-seasons/Game_of_Thrones_Script.csv\u0026#39; df = pd.read_csv(input_file_path,nrows=5) df.head() Release Date Season Episode Episode Title Name Sentence 2011-04-17 Season 1 Episode 1 Winter is Coming waymar royce What do you expect? They\u0026rsquo;re savages. One lot s\u0026hellip; 2011-04-17 Season 1 Episode 1 Winter is Coming will I\u0026rsquo;ve never seen wildlings do a thing like this\u0026hellip; 2011-04-17 Season 1 Episode 1 Winter is Coming waymar royce How close did you get? 2011-04-17 Season 1 Episode 1 Winter is Coming will Close as any man would. 2011-04-17 Season 1 Episode 1 Winter is Coming gared We should head back to the wall. We only require the columns Name and Sentence to address our problem statement. However, I am curious about finding efficient ways to read a dataset. Further, I do hope to extend the analysis with the other variables.\nPandas use an enum-like structure for a category, which allows saving on storage and computation. Well, it\u0026rsquo;s more complicated than an enum, but the comparison helps my understanding. Similarly, the string datatype is a good choice when we wish to do string manipulations from within a data frame.\ndtype = {\u0026#39;Episode Title\u0026#39; : \u0026#39;category\u0026#39;, \u0026#39;Name\u0026#39;: \u0026#39;category\u0026#39;, \u0026#39;Sentence\u0026#39;: \u0026#39;string\u0026#39;} df = pd.read_csv(input_file_path, parse_dates=[\u0026#39;Release Date\u0026#39;], dtype=dtype, converters={\u0026#39;Season\u0026#39;: lambda x: int(re.sub(\u0026#39;.*\\D\u0026#39;, \u0026#39;\u0026#39;, x)), \u0026#39;Episode\u0026#39;: lambda x: int(re.sub(\u0026#39;.*\\D\u0026#39;, \u0026#39;\u0026#39;, x))} ) df.head() Release Date Season Episode Episode Title Name Sentence 2016-05-01 6 2 Home NaN You leave the fighting to the little lords, Wy\u0026hellip; 2016-05-01 6 2 Home NaN Well, he\u0026rsquo;s never going to learn to fight becau\u0026hellip; 2016-05-22 6 5 The Door NaN Wylis! What\u0026rsquo;s the matter? A quick search leads us to Old Nan.\nOld Nan is an elderly woman living in Winterfell. She is a retired servant of House Stark known for her tale-telling abilities. She has entertained the children of Eddard and Catelyn with stories throughout their childhoods.\nOld Nan is falsely parsed as null by pandas. Now we can\u0026rsquo;t have that, can we? Let us fix it.\ndf.loc[:, \u0026#39;Name\u0026#39;] = df.Name.cat.add_categories(\u0026#34;Nan\u0026#34;) df.loc[:, \u0026#39;Name\u0026#39;].fillna(\u0026#34;Nan\u0026#34;, inplace=True) We end up with the data frame with following data types.\ndf.dtypes Release Date datetime64[ns] Season int64 Episode int64 Episode Title category Name category Sentence string dtype: object Doing the Analysis # Who spoke the most? # We can quickly find the top 10 characters according to the number of lines they had in the complete series.\ntop_10 = df[[\u0026#39;Name\u0026#39;]].value_counts().head(10).reset_index() top_10 index Name 0 0 tyrion lannister 1760 1 jon snow 1133 2 daenerys targaryen 1048 3 cersei lannister 1005 4 jaime lannister 945 5 sansa stark 784 6 arya stark 783 7 davos 528 8 theon greyjoy 455 9 petyr baelish 449 Tyrion Lannister rocks the top, followed by Jon Snow. Let us make a quick plot for our satisfaction.\ntop_10.plot(x=\u0026#39;Name\u0026#39;,kind=\u0026#39;barh\u0026#39;, title=\u0026#39;Top 10 Characters with lines\u0026#39;, xlabel=\u0026#39;Character\u0026#39;, ylabel=\u0026#39;No. of Lines\u0026#39; ) We have our guy. Now comes the question,\nWhat words did he speak the most? # We will extract Tyrion\u0026rsquo;s lines to a different data frame.\ntyrion_lannister = df.loc[df.Name==\u0026#39;tyrion lannister\u0026#39;,\u0026#39;Sentence\u0026#39;] We will do some quick string manipulations to split the lines into words.\ntyrion_lannister = tyrion_lannister. \\ str.replace(\u0026#39;[,.?!-]\u0026#39;,\u0026#39;\u0026#39;, regex=True). \\ str.lower(). \\ str.split() tyrion_lannister.head() 145 [mmh, it, is, true, what, they, say, about, th... 147 [i, did, hear, something, about, that] 149 [and, the, other, brother] 151 [there\u0026#39;s, the, pretty, one, and, there\u0026#39;s, the,... 153 [i, hear, he, hates, that, nickname] Name: Sentence, dtype: object We have a list under Sentence after splitting. A list is not so lovely inside a data frame. So let us explode it into long-form data.\ntyrion_lannister = tyrion_lannister.explode(\u0026#39;Sentence\u0026#39;) tyrion_lannister.value_counts().head(10) the 1094 you 864 i 772 to 771 a 605 of 513 and 435 my 307 it 290 me 276 Name: Sentence, dtype: int64 Oh my, the top spoken words are all stop words. To give a quick rundown, stop words make sense in a sentence but are nonsensical without context. Since we are looking at individual words, it is safe to remove them.\nI have tried four libraries-wordcloud, Scikit-Learn, Natural Language Toolkit (NLTK), and spaCy. I liked the output from wordcloud the best. So let\u0026rsquo;s have a look.\nfrom wordcloud import STOPWORDS tyrion_lannister[~tyrion_lannister.isin(STOPWORDS)].value_counts().head(10) will 139 know 109 one 100 father 83 well 73 want 72 good 66 yes 58 time 56 man 56 Name: Sentence, dtype: int64 Now that is a lot better. We already have the words of Tyrion. Nevertheless, let us now get right into making a word cloud for our visualization. First, we will have to create a word cloud object.\nfrom wordcloud import WordCloud tyrion_wc = WordCloud( background_color=\u0026#39;white\u0026#39;, max_words=2000, stopwords=STOPWORDS ) Next comes the job of adding our words into the word cloud instance.\ntyrion_wc.generate(\u0026#39; \u0026#39;.join(tyrion_lannister.values)) We are left with the visualization, and we\u0026rsquo;re done.\nplt.imshow(tyrion_wc, interpolation=\u0026#39;bilinear\u0026#39;) plt.axis(\u0026#39;off\u0026#39;) plt.show() Well, on first look, I have been tempted to make claims on the strength of Tyrion\u0026rsquo;s relationships with his family and others in the realm. Alas, but there ought to be better numerical ways to make such interpretations. Jumping to conclusions is never a good idea when we\u0026rsquo;re yet to dig out the good info from the data to back the claim. We will see if I can figure that part.\n","date":"24 November 2022","externalUrl":null,"permalink":"/posts/got-wordcloud/","section":"Posts","summary":"","title":"Game of Thrones: Who spoke the most, and what?","type":"posts"},{"content":"","date":"24 November 2022","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"I had a fresh install of cuda-tools from the Manjaro software repository and I simply couldn\u0026rsquo;t run cuda-gdb. I was getting the following error\ncuda-gdb: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory A related error was as follows:\ncuda-gdb: error while loading shared libraries: libncursesw.so.5: cannot open shared object file: No such file or directory I had to investigate the files libtinfo.so.5 and libncursesw.so.5. It was a simple version-conflict with ncurses. My distro had ncurses 6.2, whereas cuda-gdb was insisting on ncurses 5. A simple symlink was in order, actually two. The following two lines saved my day!\nsudo ln -s /usr/lib/libncursesw.so.6.2 /usr/lib/libtinfo.so.5 sudo ln -s /usr/lib/libncursesw.so.6.2 /usr/lib/libncursesw.so.5 I got the necessary pointers from this stackoverflow answer for an unrelated Android Studio question.\n","date":"8 July 2021","externalUrl":null,"permalink":"/posts/cuda-gdb-libtinfo/","section":"Posts","summary":"","title":"cuda-gdb Error with libtinfo.so.5","type":"posts"},{"content":"","date":"8 July 2021","externalUrl":null,"permalink":"/tags/troubleshooting/","section":"Tags","summary":"","title":"Troubleshooting","type":"tags"},{"content":"","date":"5 July 2021","externalUrl":null,"permalink":"/tags/cuda/","section":"Tags","summary":"","title":"CUDA","type":"tags"},{"content":"","date":"5 July 2021","externalUrl":null,"permalink":"/tags/optimization-algorithm/","section":"Tags","summary":"","title":"Optimization Algorithm","type":"tags"},{"content":"Parallel Enhanced Whale Optimization Algorithm (Parallel WOAmM) is a GPU implementation of the WOAmM metaheuristic optimization algorithm in CUDA. I chose to go ahead with an embarrassingly parallel solution modeling individuals of the population as a CUDA thread. Hence I had to give up on some fraction of the data dependencies in the original sequential algorithm. Also, parallelizing a stochastic algorithm meant I had to be careful to ensure thread safety. The size of CPU random number generators (RNGs) is a constraint for the caches of GPU, and the GPU RNGs are of lower quality. To overcome the comparatively inefficient optimization with GPU RNGs, I tried the following approaches:\nRunning multiple instances of Parallel WOAmM under CUDA blocks in parallel Increasing the number of iterations of Parallel WOAmM The final experiment varied all combinations of the parameters given in the table to find the best combination of optimization and speedup.\nFunction RNG # Blocks # Iterations Sphere MTGP32 1 30 Rosenbrock MRG32k3a 2 100 Rastrigin Philox_4x32_10 4 300 Griewank 6 Note: Each combination was run 50 times to achieve a sample size of 50, and all comparisons were between sequential WOAmM with a population size of 32 and 30 iterations. Parallelization Novelties # The main focus of the course was on parallelization, so I had to come up with novel ways to parallelize. They are as follows.\nThe population data and the fitness values were stored in the thread-local memory. Fixing the population size of the Parallel WOAmM instance to that of the CUDA warp size, i.e., 32, allowed me to use the warp level primitives to share data between threads and altogether avoid the shared memory. Butterfly reduction was used to find the individual with the best fitness. Clever use of pointer array with conditionals for the index to avoid warp divergence. Results # We find that MRG32k3a GPU RNG with 100 iterations and four blocks of GPU threads gives the best optimization and speedup.\nFuture Work # Running multiple instances of Parallel WOAmM within a single block and syncing the best solution across all instances halfway through the optimization. Replacing RNG with chaotic maps Acknowledgment # Parallel WOAmM is the end-term project I did for the Parallel Programming course instructed by Sathish Vadhiyar. I am grateful for his guidance and support. I have used the GPU node of CDS Turing Cluster, with NVIDIA Tesla K40M GPU and Xeon E5 2620 V2 CPU with 24 GB memory for my experiments.\nReferences # Paper (PDF) WOAmM WOA ","date":"5 July 2021","externalUrl":null,"permalink":"/posts/parallel-woamm/","section":"Posts","summary":"Parallel WOAmM is a GPU implementation of the WOAmM metaheuristic optimization algorithm in CUDA.","title":"Parallel Enhanced Whale Optimization Algorithm","type":"posts"},{"content":" Recently I have been trying to up my game in programming and had started reading this book. I came across an exercise that I thought was cool—so sharing it here. The question is this, given three inputs - d (day)m, (month), and y (year), find the day of the week it falls on. And they go on to give a neat formula for the Gregorian calendar.\n$$ y_0 = y - (14 - m) / 12 $$ $$ x = y_0 + y_0 / 4 - y_0 / 100 + y_0 / 400 $$ $$ m_0 = m + 12 * ((14 - m)/ 12) - 2 $$ $$ d_0 = (d + x + (31 * m_0)/12) \\mod 7 $$Well, it is pretty easy to do the calculations when you have the formula. But for me, the catch was that the formula gives an integer \\(\\in [0-6]\\) as output. My first reaction was to use conditionals. But that\u0026rsquo;s naive, and this particular section bars me from using conditionals. And it dawned on me, it is merely straightforward to use a string array of weeks and use the output given by the formula as the index for it.\nSo here goes my simple python implementation:\nAfter you run the code it may take a minute to get started! def dayOfWeek(d, m, y): y_0 = y - (14 - m) // 12 x = y_0 + y_0 // 4 - y_0 // 100 + y_0 // 400 m_0 = m + 12 * ((14 - m)// 12) - 2 d_0 = (d + x + (31 * m_0)//12) % 7 days = (\u0026#34;Sunday\u0026#34;, \u0026#34;Monday\u0026#34;, \u0026#34;Tuesday\u0026#34;, \u0026#34;Wednesday\u0026#34;, \u0026#34;Thursday\u0026#34;, \u0026#34;Friday\u0026#34;, \u0026#34;Saturday\u0026#34;) print(days[d_0]) dayOfWeek(1,3,2021) Feel free to play with the code, it\u0026rsquo;s editable!\n","date":"2 March 2021","externalUrl":null,"permalink":"/posts/what-day/","section":"Posts","summary":"","title":"Determining the Day of the Week (Choose Your Coding Language!)","type":"posts"},{"content":"","date":"2 March 2021","externalUrl":null,"permalink":"/tags/programming/","section":"Tags","summary":"","title":"Programming","type":"tags"},{"content":"","date":"16 February 2021","externalUrl":null,"permalink":"/tags/poem/","section":"Tags","summary":"","title":"Poem","type":"tags"},{"content":"A piece that came to me when I was alone at the break of the dawn. Depression is hard on its own; it is more challenging when people around you want help, but their support idea is not what you need.\nWhen I look inside of me,\nI see a child.\nI can hear him crying. Wailing!\nMy heart yearns to console him,\nto say it\u0026rsquo;s alright,\nthis is just a phase\u0026hellip;\nAlas! all I feel is numbness,\neating me from the inside.\n","date":"16 February 2021","externalUrl":null,"permalink":"/posts/child-inside-me/","section":"Posts","summary":"","title":"The Child Inside Me","type":"posts"},{"content":"I\u0026rsquo;m passionate about leveraging data and technology to solve complex problems and create meaningful insights. Currently, I\u0026rsquo;m pursuing an MBA in Business Analytics at BITS Pilani, where I\u0026rsquo;m expanding my quantitative skills to excel in data-driven decision making. My academic journey also includes a Master\u0026rsquo;s degree in Computational Biology from IISc Bengaluru and a Bachelor of Science in Biological Sciences from Bangalore University (BU). Throughout my schooling, I consistently excelled, achieving outstanding scores.\nResearch \u0026amp; Project Experience # My drive to apply knowledge extends beyond academics. I actively participate in research projects that hone my analytical and problem-solving skills. Notably, I developed a computational model of bacterial life cycles during an internship at Dr. Samay Pande\u0026rsquo;s Lab at IISc. Currently, I\u0026rsquo;m involved in a cutting-edge project building a Question Answering System using Generative AI models. Beyond research, I actively participate in collaborative projects. For example, I led a data visualization project at BITS Pilani, showcasing strong project management and teamwork abilities.\nProject Highlights # My academic and research pursuits provided opportunities to apply my knowledge and skills through various projects. Here are some that exemplify my diverse skillset:\nResearch Question Answering System using RAG Models (Generative AI) (Ongoing) # This cutting-edge project involves building a system that can answer questions based on research articles. My contributions include data collection, model construction using RAG architecture, and deployment for practical use.\nData Visualization Project # Led a team of 7 in collecting, cleaning, and analyzing student response data (204 responses) using Power BI. Effectively communicated data-driven insights through a mid-term review, demonstrating both technical and communication skills.\nWomen\u0026rsquo;s Ice Hockey Performance Modeling and Strategic Insights Project # This project combined my analytical skills with strategic thinking. I utilized multiple regression analysis to assess the impact of key variables on hockey performance. Additionally, I simulated strategic improvements and predicted their overall team point impact. My leadership ensured successful team collaboration throughout the project.\nParallel Meta-heuristic Optimization # This project highlighted my expertise in performance optimization and cutting-edge technology. I achieved a remarkable 40x speedup in algorithm execution by implementing parallel GPU architecture. Furthermore, I diagnosed limitations and showcased a keen analytical mind in addressing computational challenges.\nLeadership \u0026amp; Initiative # My drive to contribute extends beyond academics and research. I\u0026rsquo;ve actively sought leadership roles and initiatives that allowed me to develop my communication, collaboration, and problem-solving skills. Here are some highlights:\nStudent Faculty Council - Core Member # Led collaborative policy drafting across departments, ensuring alignment with organizational goals. Led impromptu seminars to address productivity concerns, showcasing my problem-solving and leadership abilities.\nPersonal Assistant to Dean at AUGSD, BITS Pilani # Planned and conceptualized advertisement videos to promote the BITSAT Exam, demonstrating initiative and creative thinking. Increased outreach efforts by organizing promotional events at schools after identifying lower student representation from the northeast region.\nTech Geeks Club - Core Member # Developed and maintained a website for the management fest, enhancing the online presence and user experience for participants. This experience honed my technical skills and commitment to user-centric design.\nI\u0026rsquo;m constantly seeking opportunities to learn and grow. Feel free to connect with me to discuss data science, technology, or anything that piques your interest!\nLinks # LinkedIn GitHub Resume ","externalUrl":null,"permalink":"/about/","section":"Home","summary":"","title":"About","type":"page"}]