The data science field is facing a challenging paradox. Big data is changing our world in many wonderful ways. Companies are offering very lucrative compensation packages to entice qualified data scientists to join their teams.
There are a number of opportunities for data developers. They are needed with helm repository and other Docker development projects. Docker is used for creating many big data environments, so data scientists with proficiency in it can have a huge advantage in the job market.
Unfortunately, despite the massive job security and lucrative salaries and benefits, companies are having difficulty finding qualified data scientists.
There appear to be many reasons for the looming shortage of data scientists. One of the biggest explanations is the fact that academic institutions are not making a big push to encourage students to pursue careers in data science. The pipeline of data scientists will be insufficient until this is rectified.
Data scientist shortage is a cause for concern
The issue is not that the supply of data scientists is not increasing. The problem is that the supply is not keeping up with demand. A couple of years ago, IBM published a report showing that the demand for data scientists would increase 28% between 2017 and 2020. The Bureau of Labor Statistics published a somewhat more conservative estimate. BLS experts reportedthat employment for related professionals would increase by 16% between 2018 and 2028. Although this estimate is lower than the IBM forecast, it is still much faster than the average of all occupations.
Companies are going to continue to struggle to find qualified data professionals as demand grows. Only an investment in data analytics education will address this problem.
Academia must make data science education a focus to overcome the shortage
Sejuti Das, an author with Analytics India Magazine recently talked about the shortage of data scientists. Das pointed out that one of the biggest causes was the lack of decent data science programs.
“Although several individuals are pursuing data science as a career, there aren’t merely enough skilled labour to fill in the jobs. The number of STEM graduates every year isn’t in sync with the market requirement of data scientists in the industry.”
He mentioned that colleges have only started offering program specific to data science over the last few years. Only a minority of students that have enrolled in these programs have had a chance to graduate thus far. Many of them don’t complete their programs, because they don’t get the support and guidance that they need.
“With colleges and universities recently started to provide full-time data science courses since the last few years, and therefore the subject hasn’t been a mainstream career choice for a long time… Experts also believe that only half of the enrolled learners manage to finish an online course, which could be attributed to their lack of guidance,” Das continues.
Unfortunately, the novelty of these programs is not the only issue. The quality of data science programs and attempts to market them are also creating issues.
Das pointed out that many universities are offering online courses in data science, which cover complex topics like Hadoop administration and Docker development. Unfortunately, many of these courses aren’t accredited and don’t meet the requirements to receive certifications.
Colleges also have not actively advertised data science positions. Unfortunately, the problem starts even earlier. Many high school counselors are not aware of some of the new technological developments that have occurred over the past decade. They often don’t even realize that the field of data science is so vast. Therefore, they are not actively encouraging students to consider majoring in data science when they go to college.
This can be seen by looking at the surplus of students in some majors, compared to the shortage of data science students. Around 19% of all college students are majoring in business. The reality is that business majors struggle to find promising jobs that require a degree because the job market is so saturated with them. Another 9% of college students receive degrees in the humanities and liberal arts. The underemployment rate for these majors is around 50%.
Colleges and school counselors should be pushing for many of the students to consider other career options. Data science should be at the top of the list. This would help reduce the shortage of data scientists, as well as the number of college graduates struggling to find work in their field of choice.
What is the short-term solution?
Over the long term, employers are going to be dependent on colleges and universities to fill the void in data science jobs. Unfortunately, it might take time for them to educate enough students to fill the vacancies.
In the meantime, employers are going to need to step up to the plate. They are going to have to train computer science and IT professionals to transition into roles as data scientists. They may not develop the same breadth of competencies as students that made certain data science, but they will be able to function in the data science field.