Wednesday, March 29, 2017

Three Benchmarks for SQL Coverage in HiBench Suite ― a Bigdata Micro Benchmark Suite

HiBench Suite is a bigdata micro benchmark suite that helps evaluate different big data frameworks in terms of speed, throughput and system resource utilization.  There are totally 17 workloads in HiBench. The workloads are divided into 6 categories:
  • Micro
  • Machine learning
  • Sql
  • Graph
  • Websearch 
  • Streaming
In this article, we will focus on one category Sql which includes 3 benchmarks:
  • Scan 
  • Join
  • Aggregate

MapReduce vs. Parallel Database Systems


To perform large-scale data analysis, there are two different approaches:[2]
  • MapReduce (MR)
  • Parallel Database Systems[4]

These two classes of systems differ in several key areas:
  • Conform to a well-defined schema or not
    • All DBMSs require that data conform to a well-defined schema, whereas MR permits data to be in any arbitrary format. 
    • MR makes a commitment to a “schema later” or even “schema never” paradigm. 
      • But this lack of a schema has a number of important consequences:
        • Parsing records at run time is inevitable, in contrast to DBMSs, which perform parsing at load time. 
        • It makes compression less valuable in MR and causes a portion of the performance difference between the two classes of systems
        • Without a schema, each user must write a custom parser, complicating sharing data among multiple applications.
  • How each system provides indexing and compression optimizations, programming models, the way in which data is distributed, and query execution strategies

SQL Benchmarks


There are three benchmarks for SQL coverage provided by HiBench Suite:
  • Scan
  • Join
    • Consume two data different sets and join them together in order to find pairs of Ranking and UserVisits records with matching values for pageURL and destURL
    • Stresses each system using fairly complex operations over a large amount of data. 
      • The performance results are also a good indication on how well the DBMS’s query optimizer produces efficient join plans. 
    • Because the MR model does not have an inherent ability to join two or more disparate data sets, the MR program that implements the join task must be broken out into three separate phases. Each of these phases is implemented together as a single MR program in Hadoop, but do not begin executing until the previous phase is complete.
  • Aggregation
    • Requires the system to scan through the entire data set, the run time is always bounded by the constant sequential scan performance and network re-partitioning costs for each node.

These workloads are developed based on SIGMOD 09 paper and HIVE-396. It contains Hive queries (Aggregation and Join) performing the typical OLAP queries described in the paper. Its input is also automatically generated Web data with hyperlinks following the Zipfian distribution.

MapReduce 


To conclude, we have summarized the findings from [2] below:
  • Start-up costs (which impacts execution time)
    • Short-running queries (i.e., queries that take less than a minute)
      • Hadoop’s increased start-up costs as more nodes are added to the cluster, which takes up a proportionately larger fraction of total query time for short-running queries.
        • As such, the desirable approach is to use high-performance algorithms with modest parallelism rather than brute force approaches on much larger clusters.
      • The authors found that their MR programs took some time before all nodes were running at full capacity. 
        • On a cluster of 100 nodes, it takes 10 seconds from the moment that a job is submitted to the JobTracker before the first Map task begins to execute and 25 seconds until all the nodes in the cluster are executing the job. 
        • This coincides with the results in [9], where the data processing rate does not reach its peak for nearly 60 seconds on a cluster of 1800 nodes
        • As the total number of allocated Map tasks increases, there is additional overhead required for the central job tracker to coordinate node activities. Hence, this fixed overhead increases slightly as more nodes are added to the cluster 
      • Enabling the JVM reuse feature in the latest version of Hadoop improved their results for MR by 10–15%. 
        • Note that this is for MR1
        • In YARN (or MR2), the tasks run in a dedicated JVM
    • Longer data processing tasks
      • This fixed cost is dwarfed by the time to complete the required processing
  • Compression
    • Hadoop and its underlying distributed filesystem support both block-level and record-level compression on input data. The authors found, however, that neither technique improved Hadoop’s performance and in some cases actually slowed execution. It also required more effort on their part to either change code or prepare the input data. It should also be noted that compression was also not used in the original MR benchmark [9]. 
  • Data loading
    • MR’s simplified loading process did make it much easier and faster to load than with the DBMSs.
  • Execution strategies
    • MR systems use a large number of control messages to synchronize processing, resulting in poorer performance than with the DBMSs due to increased overhead.
  • Failure model
    • While not providing support for transactions, MR is able to recover from faults in the middle of query execution in a way that most parallel database systems cannot.
  • Ease of use
    • In general, the authors found that getting an MR program up and running with Hadoop took less effort than with the other systems. 
    • The authors argue that although it may be easier to for developers to get started with MR, maintenance of MR programs is likely to lead to significant pain for applications developers over time.
  • Sharing data among applications (or not)
    • When no sharing is anticipated, the MR paradigm is quite flexible. 
    • If sharing is needed, however, then the authors argue that it is advantageous for the programmer to use a data description language and factor schema definitions and integrity constraints out of application programs. This information should be installed in common system catalogs accessible to the appropriate users and applications.
  • Pig and Hive[8]
    • To alleviate the burden of having to re-implement repetitive tasks, the MR community is migrating high level languages on top of the current interface to move such functionality into the run time.
  • Other tuning tips
    • The authors found that certain parameters, such as the size of the sort buffers or the number of replicas, had no affect on execution performance, whereas other parameters, such as using larger block sizes, improved performance significantly
    • The fact that MR does not transform data on loading precludes various I/O optimizations and necessitates runtime parsing which increases CPU costs

References

  1. HiBench Suite
  2. A Comparison of Approaches to Large-Scale Data Analysis
  3. HIVE-396
  4. Parallel database systems 
    • All share a common architectural design
    • including Teradata, Aster Data, Netezza, DATAllegro (and therefore soon Microsoft SQL Server via Project Madison), Dataupia, Vertica, ParAccel, Neoview, Greenplum, DB2 (via the Database Partitioning Feature), and Oracle (via Exadata).
  5. A Benchmark for Hive, PIG and Hadoop
  6. How to check default hdfs block size?
    • hdfs getconf -confKey dfs.blocksize
  7. Programming Hive
  8. Difference between Pig and Hive-The Two Key Components of Hadoop Ecosystem
  9. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI ’04, pages 10–10, 2004.

57 comments:

Blogger said...

Quantum Binary Signals

Get professional trading signals sent to your mobile phone every day.

Follow our signals right now & make up to 270% a day.

Smith said...

norton.com/setup Online Help – Step by Step guide for Norton Setup, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Setup Norton product.norton.com/setup

Smith said...

Enter Key for office.com/setup, after purchasing Office from visit office.com/setup, sign in to your www.office.com/setup account then enter product key for Office Setup.office.com/setup

Smith said...

Online Supports...
microsoft outlook support
adobe support
aol mail support

paul william said...

www.mcafee.com/activate
www.mcafee.com/activate
www.mcafee.com/activate
roku.com/link

ajay kumar said...

Norton.com/setup Online Help – Step by Step guide for Norton Setup, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Setup Norton product.
norton.com/setup

office.com/setup Online Help – Step by Step guide for Office Setup, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Setup Office product.
office.com/setup

mcafee.com/activate Online Help – Step by Step guide for mcafee activate, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Setup mcafee product.
mcafee.com/activate

Pogo Support - Call 1-844-456-8733 (US/CA) or 01444 39 0119 (UK) for Pogo-related issues. Any errors or issues will be fixed by us related to Pogo Games.
pogo support


Norton Reinstall – Norton has developed a Norton reinstall and removal tool to provides the ease of Un-installation and Re-installation. The Norton Antivirus offers you the protection from the malware, viruses, and other online threats like identity theft and data theft while shop online.
Norton Reinstall

Norton.com/setup Online Help – Step by Step guide for Norton Setup, Download & complete installation online. We are providing independent support service if in case you face problem to activate or Setup Norton product.
norton.com/setup

Norton Support said...

www.McAfee.com/Activate

Protect Your Computer , Network, Social Media Account And all other from hackers, infection ,Virus and other online threats, Mcafee Total Protection Software very Important to Activate or Setup with the Official Link mcafee.com/activate

McAfee.com/Activate

Norton Support said...

McAfee Support

McAfee Customer Service

McAfee Antivirus Support

McAfee Technical Support

McAfee Help

McAfee phone number

McAfee Customer Support

Norton Support said...

www.Webroot.com/Safe

Need to Have Advanced Technology internet Security software from Webroot safe Software company with the following Link webroot.com/safe that help to protect all device from virus, malware and other online threats.

Webroot.com/Safe

Norton Support said...

www.Webroot.com/Safe

Need to Have Advanced Technology internet Security software from Webroot safe Software company with the following Link webroot.com/safe that help to protect all device from virus, malware and other online threats.

Webroot.com/Safe

Norton Support said...

www.Norton.com/Setup
Go To Norton.com/Setup For Register Norton Account
Norton Internet Security Software is the Highly Important Software to have on the Devices, Like Computers, LAPTOPS, IPHONE, IPAD, TABLET , SMART PHONES, APPLE computers to give the Best Protection To Setup Norton easily with the Link Norton.com/Setup
http://www.norton-setupcom.com/

Norton.com/Setup

Norton Support said...


Go to www.Avg.com/Retail to Login Account


Avg.com/Retail
Get protection from malware , Intrusion, Trojan , cyber attacks get Avg software installed with the help of link www.Avg.com/Retail or call Avg support for any technical help

www.Avg.com/Retail

Norton Support said...

Kaspersky Customer Support

Kaspersky Customer Service

Devices would be free from Virus, Malware, Trojan and other online threats
Kaspersky Activation with the link activation.kaspersky.com gives you the Complete protection , like email protection , Banking Details Protections, Sensitive Information protection, important Software Protection, Get Instant Kaspersky Support or Call Kaspersky Customer Service.

http://www.kasperskysupports.us/

Kaspersky Support

Norton Support said...

www.Webroot.com/Safe

Having Problem In Installing Your Webroot Antivirus with link www.webroot.com/safe ? Don't Worry And Call Our Webroot Support Expert To Get Instant Support.

http://www.webrootsafe.xyz/

Webroot.com/Safe

gbwhatsapp apk said...

imo for pc

Aman tripathi said...

McAfee.com/Activate
Roku.com/link

Aman tripathi said...

norton.com/setup Norton, one of the largest security products providers, has made it quite easy to protect your computer system from the malicious online activities, viruses, Trojan horses, scams and other threats. By installing a Norton setup to your device, you can be sure of the privacy of your important files as well as confidential information. Be it a business or consumer, Norton offers a special security software to suit the needs of everyone. You can choose anyone from the following: norton.com/setup

Roku & Echo Device said...

If your echo won't connect to wifi Or having any problem to setup Alexa to Wifi then Don't Worry we are here to help you just follow the simple steps which are given on our website. We'll help you to amazon echo dot wifi setup help, connect echo to wifi and amazon echo not connecting to wifi and other problems. For instant help, call us at our amazon echo help number +1-888-745-1666.
setting up the echo dot
connect echo dot to wifi

Webroot Number said...

I just want to say that all the information you have given here on is awesome.Thank you.
Webroot Support | Contact Webroot

Nidhi Kapoor said...

AT&T is an American Multinational Company that provides a wide range of products and services and have many subsidiaries. And to manage all their services and product, you need an ATT account. Once you use this account and login ATT, you will be able to add your AT&T services and manage them remotely.
www att com

Cox is a private American Company that provides digital cable television, telecommunications, and Home Automation services in the United States. They also provide a very reliable support service to their users. You can register yourself with the Cox mail and get started with their services now.cox webmail

David said...

Great Blog,thanks for sharing about valuble information.are you facing any thing in viruses and malwares.if you want to know more details then visit.
Webroot Support | Webroot Help

Hussy said...

Great Blog,thanks for sharing about valuble information.are you facing any thing in viruses and malwares.if you want to know more details then visit.Webroot Technical Support | Contact Webroot

Jameson said...

Great Blog,thanks for sharing about valuble information.are you facing any thing in viruses and malwares.if you want to know more details then visit.
Webroot Technical Support | Contact Webroot

Jenny said...

I just want to say that all the information you have given here on is awesome.Thank you.Mcafee Customer Service | Mcafee Phone Number

Jenny said...

I just want to say that all the information you have given here on is awesome.Thank you.Mcafee Customer Service | Mcafee Phone Number

sandy said...

Great Blog,thanks for sharing about valuble information.are you facing any thing in viruses and malwares.if you want to know more details then visit.Webroot Contact Number | Webroot Technical Support

Steveson said...

I found so many interesting stuff in your blog especially its discussion.Really it's great article.Keep it up.Webroot Support | Webroot Help

Mak said...

Great Blog,thanks for sharing about valuble information.are you facing any thing in viruses and malwares.if you want to know more details then visit.
Webroot Contact Number | Webroot Technical Support

Hernandez said...

I found so many interesting stuff in your blog especially its discussion.Really it's great article.Keep it up.Webroot Contact Number | Webroot Technical Support |

Echo Setup said...

To Setup Echo Dot firstly you have to Download Alexa App & Login your details. After that visit www.amazon.com/EchoSetup for Echo Dot Setup. Follow Further steps to complete process. If you have any query then please contact us at 1-888-409-8111.

Technical Support said...

eset Support provides all-round assistance for eset Antivirus. As soon as you register yourself for the use of Internet facility and get the advantage of surfing the World Wide Web, you are open to innumerable threats transmitted to your system due to online browsing sessions. Encounter perfect malware blocking with eset Antivirus. Let eset Support with its techsmarties help you protect your system and rectify the security issues. We at eset Support , have team of experts who help you or guide through all the stages of eset installation process.

www.activationeset.com

Technical Support said...

Viruses and spywares can harm your computer file system and important data so now its really really important to have good security protection which can secure your online and offline digital life. McAfee is a complete security protection suite which not only protect your PC's from viruses & spywares but also it can maintain the good speed of your operating system and increase the health of it.

Visit: McAfee Technical Support

Technical Support said...

Bitdefender Support provides all-round assistance for Bitdefender Antivirus. As soon as you register yourself for the use of Internet facility and get the advantage of surfing the World Wide Web, you are open to innumerable threats transmitted to your system due to online browsing sessions. Encounter perfect malware blocking with Bitdefender Antivirus.

Visit: my-bitdefender.com

Technical Support said...

Kaspersky Support experts can lend their hand to download, install and update Kaspersky Antivirus on your system. We can also repair all errors that may crop up while installing and configuring Kaspersky Antivirus on your PC. We can help you detect and remove malicious threats, malware and spyware by performing a quick scan on all files and folders. With our robust technology, we can destroy suspicious programs and infected files from your system. Our antivirus experts can clean all online threats, including Trojan, root kits, key loggers, and worms in just single sweep. We can optimize your computer’s speed and efficiency and also protect it from being sluggish.

Visit:activation.kaspersky.com

Technical Support said...

Webroot Safe is one of the established great Antivirus software programs. In case you own a device or device that you’re connecting it to the internet or some other tool you then must have antivirus software program.

Visit: www.webroot.com/safe

Technical Support said...
This comment has been removed by the author.
Technical Support said...

www webroot com safe is a computer security software program for Microsoft Windows users that combine software as a service cloud protection with traditional Antivirus and anti-spyware desktop technologies. Built into the suite is a two-way firewall, a registry cleaner, Email anti-spam, secure browsing, anti-phishing, and a password management service. The suite was at first discharged on July 26, 2010, as the 2011 variant of the item.

Visit: best buy webroot install

Technical Support said...

you need to visit Mcafee com myaccount. And then use the same email address that you provided when you create you account first time. After you have successfully logged in to your account you can start the download of your product online. If you need help with your account restoration process please call or live chat with us.

Visit:
McAfee Antivirus Support

Technical Support said...

also, enter your email address along with other needed details to Mcafee activate. McAfee antivirus activate keeps your computer and mobile devices safe from virus attacks.

Visit: McAfee Customer Service

Technical Support said...

Webroot Support provides all-round assistance for Webroot Antivirus. As soon as you register yourself for the use of Internet facility and get the advantage of surfing the World Wide Web, you are open to innumerable threats transmitted to your system due to online browsing sessions. Encounter perfect malware blocking with Webroot Antivirus.

Visit: install my webroot

Technical Support said...

McAfee is a well-known brand and provides security tools to protect your personal data and cloud files on Windows PC, MAC, and Mobile devices. The McAfee furnishes a wide range of various antivirus and security tools such as McAfee LiveSafe, McAfee Total Protection, McAfee All Access, McAfee Antivirus Plus, McAfee Mobile Security, McAfee Virus Scan and McAfee Internet Security.
To install and activate McAfee product on your, you must have a product key or activation code. A McAfee product key or activation code comes along with your McAfee product and also you can purchase a product key from the retail store or a retail website.

Visit: McAfee Antivirus Support


Technical Support said...

Trendmicro is a private American company that provides comprehensive internet security solutions for consumers as well as businesses with various products. These services are available for home based computers, small offices as well as large business enterprises by preventing potential dangers in real time whenever they connect in the digital space for both personal and professional purposes.

Visit:
www.trendmicro.com/bestbuy

Technical Support said...

This is a great option for PCs, Macs, Tablets as well as Smartphones. Along with antivirus features, it also protects all your login ids as well as passwords. You can install Webroot smart security easily within moments and webroot.com/safe protects you forever.

Visit: webroot safe install

Technical Support said...
This comment has been removed by the author.
Technical Support said...

Best buy is leading Retail store of Electronics devices and Software’s ,New PC & Laptop users most often choose Anti virus security from same store to Install Trend Micro best buy, users need to reach www trendmicro com bestbuy if you are using search engine like Bing,Google & Yahoo look for “Geek Squad | Trend Micro” or “www trendmicro com” . Open the website your installer file automatically start downloading Run Trend Micro Installer file to Install in your PC.

Visit: www.trendmicro.com/bestbuy


Technical Support said...

Webroot Support experts can lend their hand to download, install and update Webroot Spy Sweeper Antivirus on your system. We can also repair all errors that may crop up while installing and configuring Webroot Antivirus on your PC. We can help you detect and remove malicious threats, malware and spyware by performing a quick scan on all files and folders. With our robust technology, we can destroy suspicious programs and infected files from your system.

Visit: best buy webroot install

Technical Support said...

The Norton security package is simple to setup & install at norton.com/setup. Simply find 25-character alpha-numeric code that is written on the backside of the retail card. Here may be a sample Product Key to let you understand:
xxxxx-xxxxx-xxxxx-xxxxx-xxxxx

Visit:
norton setup and install

Technical Support said...

We would ensure that you don’t need to run around to get your computer fixed, Avast Support resolves issues using remote access technology, you just need to sit back relax and let the technician do their work.
Whenever in need we’d be happy to help you, with Avast Support unlimited plans you be able to call us as many times as you need, you would not need to pay every time you call.
With an average wait time of 60 Secs, Avast-Support has an efficient team of techies who make this possible by ensuring quick resolutions anytime and every time.

Visit: www.avast.com


Technical Support said...

Viruses and spywares can harm your computer file system and important data so now its really really important to have good security protection which can secure your online and offline digital life. AVG is a complete security protection suite which not only protect your PC's from viruses & spywares but also it can maintain the good speed of your operating system and increase the health of it.

Visit: Install AVG Internet Security

Technical Support said...

According to the License Agreement terms, “To check the legitimacy of the Software use the Rightholder reserves the right to use means to verify that you have a licensed copy of the Software.” Technical Support specialists can ask to verify the legitimacy of your software use by providing a copy of a document containing the code. If you cannot provide a copy, your software will work with limited functionality and Kaspersky Lab specialists may cancel providing technical support. Therefore, you should keep the document in a secure place.

Visit: activation.kaspersky.com

Technical Support said...

Trend Micro Support provides all-round assistance for Trend Micro Antivirus. As soon as you register yourself for the use of Internet facility and get the advantage of surfing the World Wide Web, you are open to innumerable threats transmitted to your system due to online browsing sessions. Encounter perfect malware blocking with Trend Micro Antivirus. Let Trend Micro Support with its techsmarties help you protect your system and rectify the security issues. We at Trend Micro Support , have team of experts who help you or guide through all the stages of Trend Micro installation process.

Visit: trend micro download best buy

Technical Support said...

Webroot is a private American company that provides comprehensive internet security solutions for consumers as well as businesses with various products. These services are available for home based computers, small offices as well as large business enterprises by preventing potential dangers in real time whenever they connect in the digital space for both personal and professional purposes.

Visit: webroot safe install

Technical Support said...

The Webroot Technical Support have the office to remote login to your PC/Desktop or Laptop and fix any sort of issue which you may look in the establishment procedure. Definite advances are accommodated all your instructional needs. You can call all day, every day Webroot Support Toll-free number regardless of whether you are another client and the specialized group will help you by giving the solutions to your questions.

Visit: my.webrootanywhere.com

Technical Support said...

www malwarebytes com– Malwarebytes is an enemy of malware programming for Microsoft Windows, macOS, Android, and iOS that finds and evacuates malware. Made by Malwarebytes Corporation, it was first discharged in January 2006. It is accessible in a free form, which examines for and evacuates malware when begun physically, and a paid variant, which furthermore gives booked sweeps, continuous insurance and a glimmer memory scanner.

Visit: http://www-malwarebytes.com/install/


Unknown said...

Positive site, where did u come up with the information on this posting? I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.
www.office.com/setup
www.office.com/setup
www.office.com/setup

Roku & Echo Device said...

To Setup Amazon Echo, Setup Echo Plus, Setup Echo Tap, Setup Echo Show, Setup Echo Dot, Alexa App Download Setup, Echo Dot Setup-Download AlexaApp Echo setup, www.amazon.com/echodotsetup, Connect Echo to Wifi, Amazon Alexa Troubleshooting and any other Setup of Amazon Echo Device, you have to Download the Alexa App from https://alexa.amazon.com/download alexa app. You can also Download Alexa App for mac
Download Alexa App
Alexa.Amazon.com
www.alexa.amazon.com/echosetup
www.amazon.com/echodotsetup

Lucia Mandela said...

We provide free office Setup Technical support, customer service & online support for install ms office ,Get Live Chat Help & Support
office.com/setup | | norton.com/setup | office.com/setup| mcafee.com/activate