Accessibility Links

Big Data Jobs

Latest jobs

Director of Engineering
  • Job type: Permanent
  • Location: New York
  • Salary: US$175000 - US$200000 per annum
  • Description Director of Technology 175-200K Augmented reality is the future, and it is here, hop on board! This booming start-up is seeking a driven, passionate full stack developer to come in and lead the
Solutions Engineer | Threat Intelligence
  • Job type: Permanent
  • Location: Orlando, Florida
  • Salary: Negotiable
  • Description Solutions Engineer | Threat Intelligence Glocomms is currently looking for a Senior Solutions Engineer for an up-and-coming advanced SIEM and threat intelligence vendor
Senior Sales Engineer
  • Job type: Permanent
  • Location: Boston, Massachusetts
  • Salary: US$170000 - US$200000 per annum
  • Description Senior Sales Engineer Cloud Security Start-Up Boston, MA $170,000-$200,000 Base plus Commissions & Stock! An early-stage SaaS Security Vendor in the cloud space
Sr. Front-End Engineer
  • Job type: Permanent
  • Location: New York
  • Salary: US$145000 - US$165000 per annum
  • Description Sr. Front-End Engineer Global Cybersecurity Startup New York, New York $145K-$165K + Benefits An innovative, $125million dollar backed Cybersecurity firm is looking to hire a Sr
Software Engineer - Artificial Intelligence
  • Job type: Permanent
  • Location: Berlin
  • Salary: Competitive
  • Description Our client is on the search for experienced Java developers that have had hands-on professional experience working with cloud services (Amazon EC2) and Big Data Solutions such as Hadoop and Spark
Senior Wireless Consultant - BeNeLux
  • Job type: Permanent
  • Location: Brussels
  • Salary: €80000 - €100000 per annum
  • Description Senior Infrastructure Integration Solution Consultant - Wireless Specialist. Mission/ Role purpose: The Senior Infrastructure Integration Solution Consultant has the responsibility of the
Senior Data Scientist - Machine Learning
  • Job type: Permanent
  • Location: City of London, London
  • Salary: Negotiable
  • Description The company hiring is the successful data division of an international mass media group, and they are looking to add a Senior Data Scientist with experience in Machine Learning and Deep Learning
Service Delivery Manager
  • Job type: Permanent
  • Location: Braunschweig (38104), Niedersachsen
  • Salary: £100000 - £120000 per annum
  • Description Leading Global Technology company seeks experienced service delivery manager to provide technical and strategic guidance on-site at a key client location around their SAP/BI systems
Data science engineer
  • Job type: Permanent
  • Location: Philadelphia, Pennsylvania
  • Salary: US$150000 - US$200000 per annum
  • Description Position: Data Science Engineer Salary: $150,000 - $200,000 Are you actively looking for a new role where you can display your skills in Data Engineering and Machine Learning? One of the largest

A relatively new term, big data is used to explain the collection of huge amount of information in extensive quantities, from multiple sources. Big data involves data sets that are so large and complex that traditional data processing software applications are inadequate to deal with them. Big data engineers develop, evaluate, maintain, and test big data solutions within organizations. These professionals manage and creates the tools and big data infrastructure of a company, and identify methods of getting results quickly from vast amounts of data. 
Big data can come from both digital and traditional sources, and the data can be received from internal and external sources. A big data engineer is involved in accessing, analyzing and managing massive amounts of data or information and using them to meet strategic and business objectives. 

Job Specification, Salary, Outlook
Big data specialists must be business and tech-savvy with an aptitude for statistics and mathematics. Several years of experience, and the combination of these two in-demand skills will help a big data engineer to earn an above average salary, and expertise in this area can make for a very lucrative career. For the best career prospects, it’s important to stay up to date with developments and trends in this field.  

So how much do big data engineers earn? It depends on a number of factors, but the average salary for big data engineers in the United States is $96,649 per annum, with a range of $67,928 - $154,047. Factors that affect the level of salary include job location, technical skills, on-the-job experience, and academic background. Salary and other facilities are largely dependent on your ability to deliver results and level of experience. In the UK, an average salary of £34,427 per year can be expected, with a range of £17,049 - £59,520.   

Job Description
As a big data engineer, you help companies to manage an incredibly large amount of data the organizations are bombarded with regularly. You need to analyze a vast amount of data on a daily basis, and help the employer to understand how they can use the information to stay competitive, and what the data can do for them. A big data engineer’s regular tasks include supporting and developing processes, technology and personnel that analyze, gather, organize and store information. 

Typical Responsibilities:

  • Gathering and processing raw data at scale (including web scraping, writing scripts, SQL queries, APIs etc.)
  • Working closely with the engineering team to integrate algorithms and innovations into production systems
  • Supporting key business decisions with the necessary analysis 
  • Processing and analyzing massive amounts of unstructured data 
  • Providing managerial and technical leadership in a team that designs and develops large-scale cluster processing systems of available data 
  • Guiding and managing the lifecycle of a Hadoop solution, including platform selection, requirements analysis, application design, technical architecture design, development, testing and deployment
  • Extracting huge volumes of data from a variety of sources
  • Exploring data using visualization, statistics and quantitative tools
  • Establishing and maintaining effective processes for updating and validating predictive models
  • Selection of appropriate techniques for statistical processes 
  • Analysis, forecasting, and modelling of utilization trends/patterns 
  • Collaboration with data strategy partners and internal stakeholders to improve efficiency 
  • Managing the innovation cycle of generating insights, conducting analyses and advocating the integration of new concepts into scalable software solutions 

Key Skills & Qualifications      
This field is relatively new, so education and training institutions are still updating and expanding their offerings and programs to support a career as a big data engineer. A degree in computer sciences, computer engineering, mathematics or statistics is a solid starting point.  A post-graduate degree can be required for highly sought after big data engineer jobs.
Big data engineers should be able to program or code with programming languages such as Python, Java, R, Ruby, Matlab, Clojure, SQL or Pig. They should understand Hadoop, MapReduce and/or Hive. 

Required job skills:

  • Experience with data structures and complex data 
  • Management capability of Hadoop cluster, with other services 
  • Solid understanding of distributed principles of computing 
  • Capability to solve operational issues with the Hadoop cluster  
  • Proficiency with HDFS, MapReduce, Hadoop v2 
  • Experience with using solutions such as Spark Streaming or Storm, and building stream-processing systems
  • Solid knowledge of big data querying tools, such as Impala, Hive, and Pig 
  • Experience with the integration of vast amounts of data from various data sources
  • Knowledge of NoSQL databases, such as MongoDB, Cassandra, HBase 
  • Experience with various frameworks and ETL techniques, such as Flume 
  • Knowledge of various messaging systems, such as RabbitMQ and Kafka 
  • Experience with big data ML toolkits such as H2O, SparkML, or Mahout 
  • Solid understanding of Lambda Architecture 
  • Experience with Hortonworks/MapR/Cloudera
  • Ability to lead, communicate and manage 
  • A highly analytical and methodical approach

Job Trends  
It is expected that the career opportunities for big data engineers will continue to increase in the next ten years. Organizations are becoming more and more aware of the value of using big data to make informed, strategic business decisions and to see the big picture.