Profil de DJ
DJ
550.00€ /j
Big Data engineer
Consultant de l’ESN Cooptalent
Disponible le :
15/06/2020
Localisation : France
Mobilité : Mobilité nationale
Localisation : France
Mobilité : Mobilité nationale
5 années d'expérience
0 missions réalisées
AwsCloudDevOpsKafka
0 missions réalisées
DJ en quelques mots
Expériences
02/11/2017– 31/07/2017 : Big Data engineer chez Schneider Electric
I joined the company to help building the company’s global, cloud-based, data lake and Big Data Platform, leveraging the services of AWS as well as other Big Data/cloud technologies to deliver the best in-class data, services and analytic platform across the group. The project is now live.
Mission :
– Help define the architecture of the solution.
– Define the conception of the full ingestion workflow taking into account all the data challenges (incremental data management in a Big Data platform).
– Implement the solution : develop , test and validate the ingestion pipeline from extraction to data visualization.
– Documentation.
– Move the solution from sandbox to the prodution – Go Live.
– Migrate the company’s transactional data from Oracle to Aws Redshift (+100T compressed data). – I also held the role of PO of an internal project aiming to build & measure KPIs using our Big Data Platform.
Technologies used :
Aws services including : EMR, EC2, S3, CloudFormation, Cloudwatch, Lambda, StepFunction, DynamoDB, Spark (Pyspark), Presto, Hive, DataBase Migration Service, Redshift, RDS (MySql), codecommit,
Others : Jira, Git, SQL, Confluence.
© Cooptalent 1/2
28/08/2016– : Apprentice Data Engineer chez Cyres
Data Processing and Behavioral Analysis with Machine Learning
Missions:
– Participation in the deployment of Hadoop in a shared big data infrastructure based on Docker – Development and technical recipe of the ingestion workload :
• Ingestion of raw data;
• Data wrangling : Pre-treatment, cleansing and enrichment of raw data; • Storage of the transformed data in target storage systems.
– Implementation of Machine Learning algorithms to determine risk profiles: • Feature engineering
• Dimentional reduction (ACP)
• Creating a model to segment profiles using unsupervised m-learning: K-means
– Providing tools for exploring and viewing data via Tableau.
– Participation in project monitoring meetings with the different stakeholders of the project. Environment: Cloudera CDH 5.9 (HDFS, Hive, Impala), Spark 2.0.2 (Streaming, ML), Nifi, Kafka, Scala, Table
28/05/2016– 26/08/2016 : Trainee Big Data Engineer: open source collaboration chez Elastic
Add docker module to Elastic’s Beats project to monitor Docker containers:
Add to Beats data shipper a new module to monitor Docker containers (collect information from containers running on servers via the docker API, index data on Elasticsearch and restitution via Kibana in the form of dashboards )
• Missions:
– Contribute and study the existing (dockbeat) based on an older version of the Elastic Beats library
– Develop the Docker module based on Dockbeat
– Perform unit, integration and system tests.
– Write and publish a post on Elastic’s blog in order to present the project
– Write the user’s guide of the module
– Merge of the project with the official repository of Elastic
– Participate in the Elastic {On} conference (March 2017 to San Francisco) to present the module and features offered to monitor Docker containers.
Technologies: The Elastic Stack 5.0 (Elasticsearch, Logstash, Kibana, Beats), Docker, Golang, Git • NOTE: Docker module officially integrated into the Elastic Beats project: https://github.com/elastic/beats/tree/ master / metricbeat / modulus / docker
Formation
30/08/2014-26/08/2017- Ingenieur à Ecole Polytechnique de l’université de Tours Langue(s) Centres d’intérêt
Arabe
Anglais Français
