Membuat dan Menjalankan Aplikasi Apache Spark dengan Intellij IDEA pada OS Windows

    Pada artikel "Apache Spark: Perangkat Lunak Analisis Terpadu untuk Big Data" telah diperkenalkan secara singkat tentang apa itu Apache Spark dan kegunaannya. Kali ini kita akan mencoba untuk membuat dan menjalankan sebuah aplikasi Spark sederhana dengan sumber data dari Hadoop yang telah di-install dengan mode standalone pada OS Windows.
Tutorial ini menggunakan Windows 10 dan Software berikut:
  1. IDE Intellij IDEA; download dari https://www.jetbrains.com/idea/
    Sebelum mulai install Spark, jalankan Hadoop yang sudah diinstall. Buat direktori di Hadoop yang akan digunakan untuk menyimpan file teks yang akan diproses menggunakan Apache Spark. Gunakan perintah berikut:
    hdfs dfs -mkdir /tmp/input
    Kemudian, copy file teks dari drive lokal ke Hadoop dan pastikan file tersebut sudah benar-benar dikopi ke direktori Hadoop yang telah dibuat. Gunakan perintah berikut:
    hdfs dfs -copyFromLocal C:\tmp\intellij-projects\TextSearch\input\borobudur.txt /tmp/input/borobudur.txt
    hdfs dfs -ls /tmp/input

Langkah-langkah Install Apache Spark pada OS Windows :
  1. Download Spark dari http://spark.apache.org/downloads.html dan pastikan untuk mengunduh Spark yang kompatibel dengan versi Hadoop yang digunakan. Pada tutorial ini digunakan Apache Spark-2.3.1 (jadi kita unduh file spark-2.3.1-bin-hadoop2.7.tgz ) dari laman download Apache Spark.

  2. Ekstrak file tersebut ke C:\spark-2.3.1-bin-hadoop2.7
  3. Unduh file winutils.exe dari https://github.com/wmwijaya/hadoop2-standalone-windows7-windows10/tree/master/bin dan taruh file tersebut di direktori C:\spark-2.3.1-bin-hadoop2.7\bin
  4. Pada Environtment Variables > System Variables OS Windows, buat System Variables SPARK_HOME dengan value C:\spark-2.3.1-bin-hadoop2.7
  5. Edit System Variable > Path dan tambahkan C:\spark-2.3.1-bin-hadoop2.7\bin
  6. Test Apache Spark: buka Command Prompt Windows, kemudian eksekusi perintah spark-shell.cmd
    Catatan: cara edit Environment Variables > System Variables OS Windows ada di Cara Sederhana Install Hadoop 2 mode Standalone pada Windows 7 dan Windows 10
Langkah-langkah membuat dan menjalankan aplikasi Spark dengan Intellij IDEA pada OS Windows :
  1. Start Intellij IDEA as Administrator
  2. Buat Maven Project dari menu File > New > Project > Maven ( silakan ikuti sesuai gambar, kemudian terakhir klik Finish ):


  3. Muat Spark Library dari menu File > Project Structure > Modules > Dependencies > + > 1 JARs or directories…
  4. Buat Java Package "textsearch" dengan klik kanan TextSearch > src > main > java > New > Package
  5. Buat Java Class "TextSearch.java" dengan klik kanan TextSearch > src > main > java > textsearch > New > Java Class
  6. Buka TextSearch.java, kemudian copy-paste source code TextSearch.java dari GitHub TextSearch
  7. Edit konfigurasi untuk menjalankan program TextSearch dari menu Run > Edit Configuration… > + > Application
  8. Jalankan aplikasi dari menu Run > Run 'TextSearch' ( hasilnya seperti pada gambar ):
Demikian, selamat mencoba!

Comments

Anonymous said…
Dell Service Center in Pune Tilak Road : We provide best desktop laptop services. All types of repairing with original parts. Both hardware and software services are available here.

Dell Service Center in Pune Deccan : We provide laptop Screen replacement, laptop motherboard repairs, laptop hard disk replacement, laptop keypad repair, laptop adapter and battery repairs, Dell any model having any issues.

Dell Service Center in Pune Viman Nagar : We provide professional support to your device. Our team offers end to end support to our customers. We have professional experts to guide you over call, visit, and fix your product; We value your money, so we ensure our customer satisfaction, home service within hours, free pick and drop service.

Dell Service Center in Kothrud Pune : We provide services like motherboard replacement or repair, laptop screen replacement or repair, laptop hinges repair or replacement, laptop keyboard and trackpad repair or replacement, battery repair or replacement, DC jack repair or replacement. We sell genuine and durable products for laptop parts replacement. We provide doorstep services.

Dell Laptop Service Center in Shivaji Nagar Pune : We provide a warranty on our repair service. We are concerned with our top technician on the issues of the laptop. We use proven techniques to remove and replace the parts, and without wasting any time, our experts deliver the product. Our customer executives are in 24 hours service of solving any queries of our clients.

Dell Service Center In Hadapsar Pune : We provide services like laptop RAM and SSD upgrade, removing password from dell laptop, installation of operating system in your laptop, virus removal and firewall, installation of anti-malware in your laptop, data recovery etc. with the specific procedure and professional advice we extract the support service for our clients.

Dell Laptop Service Center In Pune Near Katraj : We provide services like solutions for slow working laptops and desktops, virus removal, data and file backup, transfer of data to new hard drive, drive installation, removal of popup and spyware etc. We cover accidental damage support for Dell laptops, electrical surges, drop, damaged or broken LCD due to drop or fall.

Dell Service Center In Pune Near Kothrud : We provide services like optical drive repair or replacement, laptop booting problem, data recovery, laptop liquid damage control, laptop camera repair or replacement etc. We take care of our customer's product very carefully and precisely and deliver the product in the due time period.

Dell Service Center In Pune Near Aundh : We provide services like a money-back guarantee if there are any glitches during 30 days of service. Buy and sell old laptops available you can get them at reasonable rates. We are famous for our customer-friendly price which is very affordable for our clients.

Dell Service Center In Pune Near Waked : We never make our customers wait for too long; we give services at affordable prices, we provide durable spare parts, we have well-trained engineers for proper assistance, we have a great deal of warranty after service.
Unknown said…
Dell Laptop Repair in Lucknow

One of the most popular laptop brands in the world is Dell. Their laptop is the work companion for many users. Even some consider it to be the indispensable one. When an unexpected problem comes up, our life can get severely affected. So if your life is also dependent on your laptops, then read on how to seek help for your laptops when they fail you. At we have experts for Dell Laptop alone in Lucknow who can resolve your laptop related problems. Don’t hesitate to seek help as soon as you see a small problem creeping up your computer, the smaller the issue, less time it will take, and you will have a functional computer always. Call Now 8130429742

Popular posts from this blog

Apache Spark: Perangkat Lunak Analisis Terpadu untuk Big Data

Apa itu Big Data : Menyimak Kembali Definisi Big Data, Jenis Teknologi Big Data, dan Manfaat Pemberdayaan Big Data

MapReduce: Besar dan Powerful, tapi Tidak Ribet

Tutorial Python: Cara Mudah Web Scraping menggunakan Beautiful Soup

HDFS: Berawal dari Google untuk Big Data

Validitas Rapid Test Covid 19 : Accuracy vs F1-Score, Pilih yang Mana?

Teknologi Big Data Fundamental: Kelebihan Hadoop versi 2 dibanding Hadoop versi 1

Pentingnya Web Crawling sebagai Cara Pengumpulan Data di Era Big Data

HBase: Hyper NoSQL Database