End-to-end Machine Learning pipeline for Named Entity Recognition in emails with basic implementation

The pipeline architecture

Disclaimers

  1. This project was inspired by a problem I had a chance to solve in my professional career, however, the problem presented here is different and this article does not contain any code and/or solutions used in the product.
  2. The solution presented here is a simplified one. Further steps required for…

Making containerized Python streaming data pipelines leverage schemas for data validation using Kafka with AVRO and Schema Registry

Introduction

In one of my previous articles on Machine Learning pipelines, message queues were touched as an alternative to HTTP client-server architecture which is the most common way to serve ML models nowadays.

Just to refresh, here are the advantages of using queues like Apache Kafka in ML pipelines:

  • Naturally asynchronous…

Anomaly detection in transactional data can be hard but brings benefits of discovering unknowns in vast amounts of data that wouldn’t be possible otherwise.

Photo by Charles Deluvio on Unsplash

1 Introduction

Usually, people mean financial transactions when they talk about transactional data. However, according to Wikipedia, “Transactional Data is data describing an event (the change as a result of a transaction) and is usually described with verbs. Transaction data always has a time dimension, a numerical value and refers to one…

Ivan Senilov

Machine Learning Engineer and enthusiast, knowing ML power and limitations. https://www.linkedin.com/in/isenilov/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store