Apache Spark: Perangkat Lunak Analisis Terpadu untuk Big Data

       Apache Spark adalah engine ( perangkat lunak ) analisis terpadu super cepat untuk memproses data dalam skala besar; meliputi Big Data dan machine learning. Secara lebih detailnya, Apache Spark dapat didefinisikan sebagai engine ( perangkat lunak ) untuk memproses data dalam skala besar secara in-memory, dilengkapi dengan API pengembangan yang elegan dan ekspresif guna memudahkan para pekerja data dalam mengeksekusi pekerjaan-pekerjaan yang membutuhkan perulangan akses yang cepat terhadap data yang diproses, seperti halnya streaming, machine learning, maupun SQL, secara efisien.

       Apache Spark terdiri atas Spark Core ( inti ) dan sekumpulan library perangkat lunak. Inti dari Spark adalah distributed execution engine, dan API Java, Scala maupun Python disediakan sebagai platform untuk mengembangkan aplikasi ETL ( Extract, Transform, Load ) terdistribusi. Kemudian, library perangkat lunak tambahan, yang dibangun diatas inti ( core )-nya, memfasilitasi berbagai jenis pekerjaan yang berhubungan dengan streaming, SQL, dan machine learning.



Komponen Apache Spark (hortonworks.com)

       Spark didesain untuk data science dan menyediakan abstraksi yang membuat data science menjadi lebih mudah. Para data scientist ( ilmuwan data ) sering menggunakan machine learning, yaitu sekumpulan teknik dan algorithma yang dapat belajar dari data-data yang diberikan. Algorithma-algorithma ini banyak yang sifatnya iterative ( melakukan perulangan kalkulasi ), sehingga kemampuan Spark untuk menempatkan data-data yang diproses pada cache di memory, berperan sangat besar dalam peningkatan kecepatan bagi pemrosesan data yang sifatnya iterative tersebut. Kemampuan Spark ini telah menjadikan Spark sebagai engine yang ideal bagi implementasi algorithma-algorithma machine learning. Berkaitan dengan hal ini, Spark juga menyertakan Mllib, library perangkat lunak yang menyediakan implementasi algorithma-algorithma machine learning untuk teknik-teknik data science yang sudah umum, seperti Classification, Regression, Collaborative Filtering, Clustering, and Dimensionality Reduction.

       Sebagai perangkat lunak untuk memproses data dalam skala besar, Apache Spark memiliki sejumlah keunggulan, diantaranya:
  1. Kecepatan. Apache Spark mampu bekerja 100 kali lebih cepat dibanding Hadoop. Berkat penggunaan state-of-the-art DAG scheduler, query optimizer, dan physical execution engine, Apache Spark dapat mencapai performa tinggi baik dalam hal pemrosesan data yang sifatnya batch maupun streaming.
  2. Mudah Digunakan. Dapat menggunakan bahasa program Java, Scala, Python, R, dan SQL untuk mengembangkan aplikasi yang menggunakan Apache Spark. Spark menyediakan lebih dari 80 operator level tinggi yang dapat memudahkan pengembang untuk membangun aplikasi secara parallel. Apache Spark juga dapat digunakan secara interaktif dari shell Scala, Python, R, dan SQL.
  3. Memiliki Cakupan yang Luas. Apache Spark menggabungkan SQL, streaming, dan analitik yang kompleks; menyediakan setumpuk library perangkat lunak meliputi SQL dan DataFrames, MLlib untuk machine learning, GraphX, dan Spark Streaming. Pengembang aplikasi dapat menggabungkan semua library ini dengan mulus dalam satu aplikasi yang sama.
  4. Dapat dijalankan Dimana-mana. Apache Spark dapat dijalankan di Hadoop YARN, Apache Mesos, Kubernetes, dengan mode standalone maupun cluster, atau di platform cloud seperti EC2. Pada dasarnya, Spark dapat mengakses berbagai tipe sumber data seperti halnya HDFS, Apache Cassandra, Apache HBase, Apache Hive, dan ratusan sumber data lainnya.
       Sejak peluncurannya, Apache Spark telah dengan cepat diadopsi oleh perusahaan-perusahaan dari berbagai jenis bidang industri. Raksasa dunia Internet seperti halnya Netflix, Yahoo!, dan eBay telah menjalankan Spark dalam skala super besar, secara kolektif memproses data dalam hitungan petabytes pada kluster yang terdiri atas 8000 nodes ( komputer ). Spark tumbuh dengan cepat menjadi komunitas open source terbesar di bidang Big Data, terdiri atas lebih dari 1000 kontributor dan 250+ organisasi.

Berminat untuk mencoba menjalankan aplikasi berbasis Apache Spark? Silakan ikuti tutorial berikut:


keduanya dikemas secara sedernana dan straight forward.

Sumber data yang dapat diakses Apache Spark (databriks.com) 

Ref:
1. Hortonworks, "What Apache Spark Does?," https://hortonworks.com/apache/spark/ [Accessed 29 7 2018].
2. Apache, "Apache Spark," https://spark.apache.org/. [Accessed 29 7 2018].
3. Databricks, "What is Apache Spark?," https://databricks.com/spark/about. [Accessed 29 7 2018].

Comments

Unknown said…
Terimakasih atas ulasan tentang big data pada postingan blog ini, sangat mudah dimengerti.
Vijjam Wjaya said…
Sama2,
terima kasih sudah menyimak.
Semoga bermanfaat!
Codex said…
Saya ingin banyak belajar, bisa kah saya memang kontak email kakak
Vijjam Wjaya said…
silakan: wijaya1414{at_mark}gmail{dot}com
ek said…
I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…
Data Scientist Courses This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information...
Your work is very good, and I appreciate you and hopping for some more informative posts
Anonymous said…
Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm
Logistic Regression explained
Priyanka said…
Attend The Machine Learning Course Bangalore From ExcelR. Practical Machine Learning course Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Machine Learning course Bangalore.
Machine Learning Course Bangalore
Anonymous said…
very well explained .I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm
Logistic Regression explained
"...dilengkapi dengan API..." saya izin bertanya API itu apa? suatu program kah?
Unknown said…
I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.Data Analytics Course
Unknown said…
hello sir,
thanks for giving that type of information. I am really happy to visit your blog.Leading Solar company in Andhra Pradesh
360digitmgdelhi said…
You totally coordinate our desire and the assortment of our data.
data scientist course delhi
360DigiTMG said…
Standard visits recorded here are the simplest strategy to value your vitality, which is the reason why I am heading off to the site regularly, looking for new, fascinating information. Many, bless your heart!
data science training
Very awesome!!! When I searched for this I found this website at the top of all blogs in search engines.
Data Science Training

Happy to visit your blog, I am by all accounts forward to more solid articles and I figure we as a whole wish to thank so numerous great articles, blog to impart to us.
data scientist certification
i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
Data Scientist Course
I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
Data Analytics Courses in Bangalore
madhavi reddy said…
I Want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging endeavors.
Business Analytics Course in Bangalore
I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
data analytics course in bangalore
Pallavi reddy said…
i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
data scientist course in bangalore
madhavi reddy said…
I Want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging endeavors.
data science certification in banagalore
Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites!
Data Science Training in Bangalore
Great blog found to be well written in a simple manner that everyone will understand and gain the enough knowledge from your blog being more informative is an added advantage for the users who are going through it. Once again nice blog keep it up.

data analytics courses in bangalore with placement
Honestly speaking this blog is absolutely amazing in learning the subject that is building up the knowledge of every individual and enlarging to develop the skills which can be applied in to practical one. Finally, thanking the blogger to launch more further too.

data science in bangalore
admin said…
Informasi diatas kurang lengkap ? temukan artikel terkait disalah satu web kami.

Blog Pendidikan ;
Blog Guru ;
Blog Mahasiswa ;
Blog Dosen ;
Blog Siswa ;
Blog Pelajar ;
Blog Ilmu ;
Blog Indonesia ;
Blog EDU ;

Terimakasih, semoga bermanfaat !
InstituteBlr said…
I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!
data analytics course in bangalore
Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
Data Science Course in Bangalore
Pallavi reddy said…
i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
best data science courses in bangalore

Pallavi reddy said…
i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
best data science courses in bangalore
Sekayu Ngoding said…
This comment has been removed by the author.
Fyndhere said…
Thanks for this post can you please help me to find relevant materials in nearby stores of hyderabad
Adinda said…
Good post. It’s always useful to read content from other authors and
practice something from other websites.

Visit Us
Pallavi reddy said…
I am glad to discover this page. I have to thank you for the time I spent on this especially great reading !! I really liked each part and also bookmarked you for new information on your site.
artificial intellingence training in chennai
princika said…
I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
data science training in chennai
Nice blog. Good work. Clear explanation and informative content. Keep sharing more blogs.
Artificial Intelligence Course in Hyderabad with placements
Hi,
It was great and informative while reading your article on Apache Spark, I liked the content it's very easy to understand and useful. Thanks for sharing Data Science Training in Pune
data science said…



I was basically inspecting through the web filtering for certain data and ran over your blog. I am flabbergasted by the data that you have on this blog. It shows how well you welcome this subject. Bookmarked this page, will return for extra. data science course in jaipur

Ramesh Sampangi said…
Thanks for sharing this blog with us. Good content and excellent work. Keep maintain this great work.
AI Patasala Data Science Training in Hyderabad
AI Patasala Artificial Intelligence Course in Hyderabad
Anonymous said…
I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
aws training in hyderabad
Anonymous said…
Amazing Article! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.If you are Searching for info click on given link
Data science course in pune
data science said…
Really appreciated for sharing this article. Very Informative.
If you are looking for advancement in your career, want to learn the data science process and its techniques, Visit Learnbay.co website to know details related to data science courses in Bangalore.
https://www.learnbay.co/data-science-course/data-science-course-in-bangalore/
princika said…
I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
iot course in bangalore
Thanks for the informative and helpful post, obviously in your blog everything is good..
data scientist course
Anonymous said…
Thanks for your marvelous posting! I really enjoyed reading it, you could be a great author. I will be sure to bookmark your blog and definitely will come back in the foreseeable future. I want to encourage you to continue your great posts, have a nice afternoon! online digital marketing course with placement
Remer Mirror said…
I'm extremely inspired along with your writing skills as neatly as with the format in your blog. Is that a paid subject or did you customize it yourself? Anyway stay up the excellent quality writing, it's rare to look at a great blog like this one today.. Bathroom Mirror
Techwriter said…
I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end.
Data Analytics Courses In Pune
Ms in Germany said…
Excellent goods from you, man. I have understand your stuff previous to and you are simply extremely fantastic. I actually like what you’ve obtained right here, really like what you are saying and the way in which by which you are saying it. You are making it enjoyable and you still care to stay sensible. I can’t wait to read much more from you. This is really a wonderful site. Ms in Germany
Tutorials said…
Nice article with valuable information. Thanks for sharing.

AWS Training in Chennai | AWS Training institute in Chennai
Skillslash said…
I like your post. I appreciate your blogs because they are really good. Please go to this website for the Data Science Course: For Data Science Course Data Science course in Bangalore. These courses are wonderful for professionalism.
Ramesh Sampangi said…
Fantastic blog, really nice blog, and useful to all. Informative and knowledgeable content. Thanks for sharing this blog with us. Keep sharing more stuff like this.
AI Patasala Data Science Courses in Hyderabad
Akshat said…
Your blog is very helpful. I read this blog. Thanks For Sharing With Us. I will definitely go ahead and take advantage of this. You absolutely have wonderful blogs. Cheers for sharing with us your blog. For more learning go through skillslash.
For Data Science Course Data Science course in Bangalore
I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.data science training in rohtak
PMP Course in Malaysia said…
This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea. PMP Course in Malaysia
EaglePie said…
Thanks for sharing this valuable information, we also provide instagram video download and,
really appreciate your hard work and good research that help us in for of your good valuable article. Keep updating us with your awesome content.

Dean said…
Many good points in this piece. Please check out our Instagram Reel Downloader, which is one of the best tools of its kind out there.
Dina Shaw said…
If you're looking for a place to download pornhub videos, please try the Pornhub downloader. It's free and unlimited.
Blogger Pro said…
Thanks for sharing this great article we appreciate it, we provide instagram reels download freely and unlimited.
Data Science said…
They're produced by the very best degree developers who will be distinguished for your polo dress creating.
You'll find polo Ron Lauren inside exclusive array which include particular classes for men, women.
360DigiTMG data science course
nagabhushan said…
Viably, the article is actually the best point on this library related issue. I fit in with your choices and will enthusiastically foresee your next updates. data science course in pune
Data Science said…
Data Science has understood the necessity of every scholar and ensure that every scholar gets an unmatched studying experience for the lifetime
data science institutes in hyderabad
Data Science said…
It's late finding this act. At least, it's a thing to be familiar with that there are such events exist.
I agree with your Blog and I will be back to inspect it more in the future so please keep up your act.
360DigiTMG data science course

Data Science said…
Data Science is a subject that impacts each department of business and thus gaining significance as it supplies options to many of the problems that are raised in a enterprise
data scientist training in hyderabad
Data Science said…
Any way I’ll be subscribing to your feed and I hope you post again soon. business analytics course in hyderabad
data analytics course in patna said…
Explore what the best Data Science training institute is offering you. Learn advanced technologies from the best industry experts.


<a href="https://360digitmg.com/india/data-analytics-certification-training-course-in-patna>data analytics course in patna</a>
Data Science said…
Its as if you had a great grasp on the subject matter, but you forgot to include your readers.
Perhaps you should think about this from more than one angle.
data science institutes in hyderabad
Data Science said…
Going to graduate school was a positive decision for me. I enjoyed the coursework, the presentations, the fellow students, and the professors.
And since my company reimbursed 100% of the tuition, the only cost that I had to pay on my own was for books and supplies.
Otherwise, I received a free master’s degree. All that I had to invest was my time.
data analytics course in hyderabad
Nidhi Kalyan said…
Residential Plots in Mathura Vrindavan - Find your residential plots in at the most affordable price. Reserve your plots today.
Deep Verma said…
We offer you the best Residential Plots in Mathura Vrindavan. We offer our customers more than just satisfaction.
Divya Mani said…
Lassi n' Shakes is the biggest and fastest growing Top 10 Famous Food Franchise in India. You not just get delicious and hygienic food prepared at your doorstep but also earn from a business of your choice to start with minimum investment.
REKHA SINGH said…
BusinessBooks are the leading supplier of outsourced tax agent and Account outsourcing services in Melbourne, Australia. As a company who have a true understanding of what it takes to grow your business, BusinessBooks know the importance of providing a service that is effective whilst being efficient.
Amit Kumar said…
Home Extensions Melbourne, Australia - Bullseye Home Builders are experts in building extensions to your home and are one of the top service providers for your home extension needs.
Ajay said…
Learn Ethical Hacking in Hindi from professionals. Get the best ethical hacking course for your level.
Chirag Grover said…
At familyroomcafe, we want your dining experience to be the best it can be. Book a table with us and enjoy our delicious dishes!
Genzmmo said…
Thanks for sharing this great article we appreciate it, we provide down video from facebook freely and unlimited.
Studylivezone said…
Thanks a lot for giving us such a helpful information. You can also visit our website for amity solved synopsis
Genzmmo said…
uraqt.xyz Entertainment information and creative ideas.
This comment has been removed by the author.
Yummyhome said…
If I want to introduce the best Instagram downloader site, it is undoubtedly Instro. Because in this site you can easily download reels, videos, photos, albums and all other things.

https://downloadinsta.app/

Popular posts from this blog

Tutorial Python: Cara Mudah Web Scraping menggunakan Beautiful Soup

MapReduce: Besar dan Powerful, tapi Tidak Ribet

Apa itu Big Data : Menyimak Kembali Definisi Big Data, Jenis Teknologi Big Data, dan Manfaat Pemberdayaan Big Data

Validitas Rapid Test Covid 19 : Accuracy vs F1-Score, Pilih yang Mana?

HDFS: Berawal dari Google untuk Big Data

HBase: Hyper NoSQL Database

MongoDB : Database NoSQL Berbasis Dokumen yang sedang Naik Daun di Era Big Data

Pentingnya Web Crawling sebagai Cara Pengumpulan Data di Era Big Data

Teknologi Big Data Fundamental: Kelebihan Hadoop versi 2 dibanding Hadoop versi 1