# Technology Stack



|                    |                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Database           | <p>Elasticsearch (Search engine, querying, visualizations, recommendation engine, etc)<br>Redis (In-memory database, speeding up the processing)<br>SQL</p><p>Neo4j (For relational graphs)</p>                                                                                                                                                                                                                                                                              |
| For Queue          | Redis, SQS, Rabbitmq                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| Streaming          | Kafka                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| AWS Cloud          | <p>SQS Queue<br>S3<br>Lambda Function </p>                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| Python             | <p>FastAPI (gunicorn, swagger, ReDoc and other features)</p><p>Flask (in production - with concurrency)<br>Celery (Parallel Processing)<br>Airflow (For Scheduling - Cron Job)</p><p>Selenium driver (For automation and scrapping)</p><p>Data Science (Pandas, NumPy)<br>Natural Language Processing (nltk, spacy)</p><p>Image Manipulation and Processing (opencv, numpy)</p><p>Neural Network (Keras and Tensorflow)</p><p>Data Visualisation (matplotlib, seaborn, )</p> |
| Deployment         | <p>Docker</p><p>auto-scaling (as per request traffic)</p>                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Visualization      | <p>Kibana</p><p>python (matplotlib, seaborn)</p>                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Project Management | <p>git (For version control)</p><p>pivotal tracker</p>                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Big data           | Apache pySpark                                                                                                                                                                                                                                                                                                                                                                                                                                                               |

