Tech Blog
 

Welcome to Saindavi , our three-bedroom farmhouse. It's an excellent spot to relax and unwind. Saindavi, our little paradise, is approximately 124 km from my residence in Bangalore and about a 60-minute drive from Bangalore International Airport (KIA). Located about a 10-minute drive from Saindhavi is the medieval temple complex known as Leepakshi, meaning "Get Up Bird" in English, constructed by the Vijayanagara Empire's kings. This temple complex gained additional recognition following the visit of our Honorable Prime Minister, which attracted media attention. The spiritual sites of Puttaparthi, Kadiri and Sadguru's Ashram near Chickballapura are approximately a 45-minute drive away by car. Keep an eye on this non-tech section of my blog, as I will begin to share a variety of topics that I am passionate about, beyond just technology.

Electrocardiogram (ECG) signals are pivotal in diagnosing cardiovascular conditions, as they provide critical insights into the heart's electrical activity. Despite their significance, the acquisition of high-quality, annotated ECG data poses challenges, largely due to privacy concerns, as well as substantial financial and time constraints. This MTech dissertation in Data Science and Engineering addresses these challenges by investigating the potential of generating synthetic ECG data through the application of Artificial Intelligence (AI) models, specifically Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN). Research Objectives Develop models using Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN) to generate synthetic ECG data. Compare the statistical properties of synthetic and real ECG data to ensure clinical utility. Methodology The research is divided into four phases: Detailed Literature Survey: This study employs the Python library, Litstudy, to conduct an exhaustive review of existing literature on ECG data and machine learning techniques for synthetic data generation. Litstudy facilitates the analysis by extracting metadata from various scientific sources, standardizing the data, and managing documents through filtering, selection, deduplication, and annotation. Additionally, it provides statistical analysis, generates bibliographic networks, and utilizes natural language processing (NLP) for topic discovery, making it a powerful tool for conducting detailed literature surveys and reviews. The utility program developed for this purpose will be shared through a GitHub link. Development of VAE Model:  This phase involves creating and training a Variational Autoencoder (VAE) model to generate synthetic ECG data for both normal and abnormal ECGs. The development of the VAE model comprises encoding the input ECG data into a latent space, capturing essential features, and decoding this latent representation to reconstruct the ECG signals. The model utilizes the variational inference approach to approximate posterior distributions, allowing for efficient generation of new data points. The MIT-BIH Arrhythmia Database and the PTB-XL dataset, hosted by PhysioNet, are employed for this purpose. The MIT-BIH dataset is instrumental for developing and evaluating algorithms for cardiac arrhythmia detection, ECG signal processing, and machine learning applications, while the PTB-XL dataset is a comprehensive resource that includes a diverse array of ECG recordings with detailed annotations. To ensure the quality of the generated ECG data, the NeuroKit2 library is utilized for signal processing and quality assessment. This library offers various tools for cleaning ECG signals, detecting peaks, and computing heart rate variability indices, ensuring that the synthetic data closely resembles the original data in terms of quality and statistical properties. Statistical comparisons, including Maximum Mean Discrepancy (MMD) and the Kolmogorov-Smirnov (KS) test, are conducted to evaluate the similarity between the synthetic and real data distributions. This approach addresses the challenges of obtaining high-quality, annotated ECG data and facilitates training machine learning models, enhancing their performance and robustness. Development of GAN Model:  T he development of the GAN model for generating synthetic ECG data involves several key steps: Dataset Utilization: The MIT-BIH Arrhythmia Database and the PTB-XL dataset hosted by PhysioNet are employed to provide diverse ECG recordings. These datasets are instrumental for developing and evaluating algorithms for cardiac arrhythmia detection and ECG signal processing. Training and Testing: The GAN model is trained using these datasets, focusing on generating synthetic data for both normal and abnormal ECGs. The synthetic data is then evaluated using statistical measures like Maximum Mean Discrepancy (MMD) and the Kolmogorov-Smirnov (KS) test to compare the similarity between synthetic and real data distributions. Quality Assurance: Tools like the NeuroKit2 library are utilized for signal processing and quality assessment of the generated ECG data. This ensures that the synthetic data closely resembles the original data in terms of quality and statistical properties. Evaluation and Analysis: Perform rigorous evaluations of the generated data and report findings, including statistical tests and visual comparisons to ensure data quality. Challenges and Future Work Addressing noise and artifacts in ECG data to improve the quality of synthetic data. Overcoming the scarcity of specific ECG datasets, particularly for Ventricular Tachycardias. Expanding the use of various sampling techniques in the latent space for more diverse synthetic data generation.

This blog post begins with a playful analogy to the scientific paper by King and Welling [1], likening to a friend who studies your fashion to create their own distinctive style. Most of inspiration is drawn from this paper. VAE is a generative model that learns from a sampled distribution. Your Quirky Friend: The Encoder This friend, called "Q," encapsulates the essence of your style and records it in a notebook, symbolizing the latent space in a VAE. Q uses the encoder for this task. They observe your outfit to capture its essence. Instead of merely replicating your outfit, Q identifies the key features that make your style unique. Q has several methods to capture the key features of your style using the encoder, which is a neural network and can take various forms, such as fully connected networks, Convolutional Neural Networks (CNNs) for image data, or Long Short-Term Memory (LSTM) networks for sequential data. Once Q has captured the key features of your style, they record these insights in their notebook. This notebook, or latent space, is where Q stores the essence of your fashion sense. Let's take a closer look at what this latent space represents More Concretely, the Encoder is responsible for mapping the input data into a latent space representation using neural network forms just mentioned. It takes the input data and compresses it into a smaller, latent variable space, typically characterized by a mean and variance that define a probability distribution. Q's Notebook: The Latent Space Q's notebook represents the latent space in the VAE. It's where Q stores their notes, sketches, and ideas inspired by your fashion sense. This notebook contains the essence of your style, distilled into a set of key features. In the VAE, the latent space is a probabilistic representation of the input data. It's a distribution over the possible values of the latent variables, which capture the underlying patterns and structures in the data. ELBO – Evidence Lower Bound In order to make the VAE learns a meaningful latent space representation, the model optimizes the Evidence Lower Bound (ELBO). The ELBO has two main components: the reconstruction term, which checks how well the decoder can recreate the input data from the latent space, and the regularization term, which ensures the latent space distribution matches a prior distribution (usually a standard Gaussian distribution). Reparameterization: Q's Creative Twist Now, imagine Q wants to create a new outfit inspired by your style, but with a twist. Instead of directly sampling from their notebook, Q uses a clever trick: they sample features from a standard normal distribution (like a random fashion magazine) and then transforms the sample using their notebook's parameters (like applying their fashion sense). This is similar to the reparameterization trick used in VAEs. By sampling from a standard normal distribution and transforming the sample using the latent space's parameters, we can backpropagate through the sampling process and optimize the VAE's parameters efficiently. Q's Fashion Creations: The Decoder Q will use their parameterized samples to create new outfits inspired by your fashion sense. Q's creations may not be exact replicas, but they capture the essence of your style. This is the decoder in a VAE. The decoder takes the reparameterized samples from Q's notebook and uses them to generate new data samples similar to the original input data. In the VAE, the decoder is trained to reconstruct the input data from the latent space. decoder is also another neural network that takes samples from the latent space and reconstructs the input data, mapping the latent variables back to the data space. Key Functions Encoder: Compresses input data into a latent representation. Decoder: Reconstructs data from the latent representation. Q's Fashion Evolution: Training the VAE As Q continues to study your fashion sense, create new outfits, and refine their reparameterization trick, they develop a unique understanding of your style. This process is like training the VAE. During training, the VAE learns to optimize the encoder, latent space, and decoder. The goal is to find a balance between reconstructing the input data accurately and capturing the underlying patterns and structures in the data. Training Objective The VAE is trained to minimize the difference between the original input and the reconstructed output while also ensuring that the latent space follows a desired distribution (usually a Gaussian distribution).  This structure allows the VAE to learn meaningful representations of the data while also enabling effective generation of new data samples. The Result: A Quirky yet Stylish VAE Through this process, Q develops a unique understanding of your fashion sense, which is reflected in their quirky yet stylish outfits. Similarly, the trained VAE can generate new data samples that capture the essence of the input data, while also introducing new and interesting variations. Figure 1.0 Variational Autoencoder (VAE) Architecture. The flow of data is indicated by arrows connecting each block, showing the sequence of operations from input to output Implementation Details Using Pytorch The encoder/decoder-based Variational Autoencoder (VAE) shown in this Figure 2.0 processes inputs consisting of sine waves. These sine waves have an amplitude of 1 unit and a single frequency, but the phase varies randomly within the range of [-π, +π]. The encoder maps these inputs to a latent representation, characterized by mean and variance, which the decoder uses to reconstruct the sine waves. Figure 2.0 The Encoder-Decoder architecture of a Variational Autoencoder (VAE) is designed to generate synthetic sine waves. It is important to note that the linear layers do not inherently compute the mean and variance of the input (such as averaging the input values in the case of the mean Instead, they learn to produce a vector that represents the and variance of latent distribution as part of the VAE training process VAE Loss Function The VAE loss function [1] comprises two main components: Reconstruction Term: Measures the discrepancy between the input data and its reconstructed version. Regularization Term (Kullback-Leibler Divergence): Ensures that the latent space distribution aligns closely with a prior distribution. By minimizing the VAE loss function, the model learns to encode input data into a meaningful latent space representation, decode this representation into a reconstructed version of the input data, and generate new data samples akin to the input data. Figure 3.0 This figure illustrates the VAE (Variational Autoencoder) loss function during training. The plot shows the decline in average loss over 25 epochs, highlighting the model's learning progression. The x-axis represents the epochs, and the y-axis represents the average loss, with a noticeable trend of decreasing loss as the training continues, indicating improved model performance. Generating Synthetic Data Using VAEs The steps for generation of Synthetic Sine Wave data using a Trained VAE are given here under: Set the VAE to evaluation mode after sufficient convergence. Disable gradient tracking to save memory and computation. Sample latent vectors from a standard normal distribution. This is on Q's Creative Twist mentioned above. Q's Synthetic Data Factory Powered by Circular Buffers A circular buffer, often used for efficient data management, operates like a fixed-size, looping queue. In the of a VAE system, it functions as follows: The circular buffer comprises two components—the read buffer and the write buffer: Read Buffer: This is where Q continuously inputs random fashion patterns drawn from various magazines. These patterns are stored sequentially in the buffer for processing by the VAE encoder. The read buffer ensures a steady data to the encoder enabling operation. Write Buffer Once the read buffer reaches its capacity (i.e., it becomes full), the write buffer takes over and begins storing new incoming patterns. This mechanism allows the system to handle data without overflow or interruptions. The decoder subsequently reads patterns the write buffer, transforming them into synthetic sine waves as part of the output. The circular buffer operates such that when the end of the buffer is reached, it loops back to the beginning. This ensures optimal use of space, as no data is lost or wasted—old data is simply overwritten when the buffer cycles around. In this implementation, the circular buffer enables efficient coordination between the encoder and decoder ensuring a seamless flow of and pattern decoding into synthetic sine waves. It is akin to a never-ending loop pattern generation and transformation. Video 1.0: "Circular Buffers in Action – Q's Synthetic Data Factory" - The top window showcases Q randomly sampling fashion patterns and placing them into the read buffer. Once the buffer is full, the decoder activates, reading these patterns to produce synthetic sine waves. The green sine wave represents the reference wave with zero phase. The red sine wave is the noisy synthetic output generated by the decoder, while the blue sine wave is a smoothed version of the raw decoder output. References [1] Diederik P. Kingma and Max Welling (2019), An Introduction to Variational Autoencoders, Foundations and Trends in Machine Learning, arXiv:1906.02691v3 [cs.LG] 11 Dec 2019, https://doi.org/10.48550/arXiv.1906.02691

NotebookLM offers the ability to generate Mind-maps, a powerful visual tool for organizing thoughts and presenting complex information in a structured, intuitive format. Mind-maps use diagrams to represent ideas, concepts, and their relationships, making it easier to grasp connections and foster creativity. Here’s an example of a Mind-map created using NotebookLM that summarizes my  M.Tech  thesis: By converting dense data into accessible visuals, Mind-maps like this serve as an effective medium for brainstorming, planning, and sharing knowledge. Have you explored the use of Mind-maps in your work or research?

This blog delves into the science of exploring scientific literature, showcasing how tools like Litstudy [1] and Jupyter Notebook can transform your approach to research. Drawing from my experience using Litstudy to craft the introductory chapter of my M.Tech dissertation's literature survey, this post provides an overview of its features. Designed for readers familiar with Jupyter Notebook [3], it highlights how Litstudy enables you to effortlessly navigate new research domains, analyze scientific papers, and enhance your analysis with interactive visualizations . Figure 1. 0 Software architecture of litstudy, showcasing its integration with Jupyter notebooks and Python scripts. The system uses various libraries such as bibtexparser, pandas, and gensim for metadata retrieval, analysis, and visualization. Search Functionality of Litstudy The litstudy [2] search feature aggregates metadata from scientific publications, highlighting important aspects such as the title, authors, publication date, and DOI. However, it does not offer access to the complete documents. This function enables users to obtain a detailed overview of literature databases. A DOI serves as a unique link for digital content, offering crucial metadata for reliable citations and access. The Various  literature databases and their associated functionalities supported by this library include: Scopus: Offers functions to retrieve metadata for documents using identifiers (DOI, PubMed ID, Scopus ID) or approximate title-based searches. Enables submitting queries to the Scopus API and importing CSV files exported from Scopus. Semantic Scholar: Allows metadata retrieval using various identifiers (e.g., DOI, ArXiv ID, Corpus ID). Provides functions for refining metadata, submitting queries to the Semantic Scholar API, and obtaining results. CrossRef: Offers metadata retrieval for documents using DOI with timeout settings to manage server communication. Supports querying the CrossRef API with options for sorting and filtering results. CSV: General-purpose CSV loading with options for field name customization and filtering. litsudy [2] attempts to infer field purposes or use explicitly defined fields. Other Platforms: Functions for importing metadata from specific databases, including IEEE Xplore, Springer Link, DBLP, arXiv, BibTeX, and RIS files. Each platform supports specific queries or file formats for metadata retrieval. Document handling classes: Document: Represents a single publication, storing metadata such as title, authors, abstract, DOI, and publication source. DocumentSet: Manages a collection of documents, supporting filtering, deduplication, merging, and set operations (e.g., union, intersection). Author: Represents an author, containing details like name and affiliations. PublicationSource: Defines the source of a document (e.g., Scopus, PubMed), tracking source-specific metadata. Figure 2.0 UML diagram illustrating the relationships among key classes in Litstudy, including Document, DocumentSet, Author, and Publication Source. It highlights the attributes, functionalities, and interactions within the system for handling scientific publication metadata. Fetching Documents using Search Query Figure 3.0 illustrates Python code snippets utilizing the " litstudy.search _semanticscholar" function. It showcases the retrieval of academic documents based on different search queries [z1-z3] along with corresponding document counts. Enables operations such as union (|) and intersection (&). Figure 4.0 Documents that are fetched from Semantic Scholar Create a Document Corpus from a Collection of Documents The build_corpus function prepares text data from documents for tasks like topic modeling, natural language processing, and clustering. It uses advanced preprocessing techniques (e.g., token filtering and n-gram merging) and outputs word-frequency vectors alongside a word-index dictionary, integrating seamlessly with other Litstudy features . Topic Modeling: Discovers popular topics and trends in scientific publications using NLP methods (e.g., LDA, embedding-based techniques) and generates topic lists with keywords, visualizations, and thematic insights. NMF Model: Extracts latent topics from document corpora with adjustable parameters (e.g., number of topics, iterations) to support literature reviews, research exploration, and thematic grouping. Retains important context that could be lost if words are analyzed individually N-Gram Merging: Combines frequently co-occurring tokens into meaningful phrases (e.g., bigrams, trigrams) to enhance text representation and improve accuracy in clustering and topic modeling. Topic Clouds This feature visualizes topics identified through modeling by creating customizable word clouds, where the size of the words indicates their significance, helping users intuitively understand key terms and thematic structures. It accepts trained topic models (such as NMF or LDA) and allows adjustments to parameters like font size and color schemes. These visualizations can be saved as images, used for topic labeling, and easily integrate with Litstudy pipelines and Jupyter Notebooks for interactive exploration. Figure 5.0 Word clouds illustrating key terms for various topics, highlighting the central concepts with size denoting significance. These visualizations offer a quick grasp of thematic structures and aid in topic labeling and further analysis. DocumentSet enables efficient management and analysis of document collections. The member function "best_documents_for_topic" in this class is utilized to pinpoint the most pertinent documents for a specific topic. Figure 6.0 The `litstudy.nlp.TopicModel` is a trained model designed to analyze topics and relationships within documents and tokens. It incorporates matrices to represent topic-document and topic-token mappings, along with the `best_documents_for_topic` function that identifies key documents associated with a specific topic. Plot Embedding The Litstudy Plot Embedding functionality visualizes document relationships in a low-dimensional scatter plot, where proximity indicates similarity. It uses precomputed embeddings generated by techniques like word or sentence embeddings and topic modeling Figure 7.0 A scatter plot visualizing relationships between documents in a low-dimensional space, where proximity of points reflects document similarity based on embeddings generated through techniques like word and sentence embeddings or topic modeling Citation Network This feature illustrates the citation network of scientific papers by generating a graph where nodes symbolize documents and edges indicate citation links. By utilizing a DocumentSet as input, it permits customization of node size, edge thickness, and color coding according to metadata such as publication year or source. The resulting graph can be either interactive or static, facilitating the analysis of relationships and connections among documents. Figure 8.0 A graph visualizing the citation network of scientific publications, where nodes represent documents and edges depict citation relationships. The visualization is customizable with parameters such as node size, edge thickness, and metadata-based color coding, highlighting connections and relationships between documents Summary The key components related to Litstudy for M.Tech dissertations in scientific literature research include: Comprehensive Metadata Management: Enables the retrieval and management of metadata from various literature databases such as Scopus, Semantic Scholar, and CrossRef. Advanced Text Analysis: Offers features like document filtering, n-gram merging, and corpus creation for NLP applications. Topic Modeling and Visualization: Provides tools to identify trends using LDA, NMF, word clouds, and embedding plots for thematic insights. Citation Network Analysis: Illustrates citation connections among documents with customizable graphs for detailed research linkages. Seamless Workflow Integration: Integrates with libraries like bibtexparser and pandas in Jupyter Notebooks to enhance analysis and interactivity. References S. Heldens, A. Sclocco, H. Dreuning, B. van Werkhoven, P. Hijma, J. Maassen & R.V. van Nieuwpoort (2022), "litstudy: A Python package for literature reviews", SoftwareX , 20, 101207. DOI: 10.1016/j.softx.2022.101207 . Literature Databases — litstudy 0.1 documentation Link to Jupyter Notebook available on request from Google Collab