Which of the following statements about Spark application architecture is correct?
6406531967359.
A Spark program generates a set of tasks that are grouped into stages, each of
which executes concurrently with the driver being responsible for collecting results from each
stage for final output.
6406531967360.
A Spark action generates a job, with each job being broken down into multiple
stages where a stage is a set of tasks that can be executed in parallel and stages themselves are
interdependent, with each task being the unit of work mapped to an executor.
6406531967361.
A Spark program generates a job, with each job being broken down into
multiple stages where a stage is a set of tasks that can be executed in parallel and stages
themselves are interdependent, with each task being the unit of work mapped to an executor.
6406531967362.
A Spark action generates a task that is further broken down into multiple
stages where a stage is a set of jobs that can be executed in parallel and stages themselves are
interdependent, with each job being the unit of work mapped to an executor.
6406531967363.
A Spark program generates a task that is further broken down into multiple
stages where a stage is a set of jobs that can be executed in parallel and stages themselves are
interdependent, with each job being the unit of work mapped to an executor.
A published solution is not available for this question yet.
Consider the problem of sorting a 1 petabyte file of numbers stored in a Hadoop cluster of 10
machines where the size of each HDFS block is 100MB. Which of the below methods is the most
likely to finish soonest in practice?
6406531967364.
Use quicksort on each machine and then merge each machine's sorted data on
the master
6406531967365.
Exploit duality of hashing vs sorting and write a hash function that assigns
each line in the file to a distinct machine, and then sort within each machine
6406531967366.
Write a general-purpose MapReduce / Spark program that picks the Nth
highest descending value from a set, invoke it iteratively from 1 to S (S=Number of lines in file),
and create a new file of sorted data
6406531967367.
Mergesort on each machine by limiting use of memory, and then between all
possible pairs of machines
A published solution is not available for this question yet.
A website sees 1 Billion hits every month. The website owner wants to count average hits per
customer in the latest month, where a customer is denoted by the IP address of the device. The
owner has at his disposal a Hadoop cluster of 5 workers and 2 masters each with 1GB of RAM.
Which of the following methods is the most likely to finish fastest?
6406531967368.
Write a Spark program that forms a Dataframe as grouping by IP address with
count as aggregate, followed by another stage that computes the avg on top of the Dataframe of
the first stage
6406531967369.
Write a MapReduce program where the Map does nothing useful, Combine
computes the aggregated hits per customer, Shuffle combined data based on IP address across
workers, and in the Reduce, build hash table on each machine with hash key = IP address and
hash value = counter, followed by another Reduce that finally computes the avg on top of all hash
values.
6406531967370.
Write a Spark program that forms a Dataframe as grouping by IP address with
count as aggregate, followed by a take into a list in the Spark driver which further computes the
average of all the individual counts in the list
6406531967371.
Write a MapReduce program where the Map does nothing useful, Shuffle data
based on IP address across workers, and in the Reduce, build hash table on each machine with
hash key = IP address and hash value = counter, followed by another Reduce that finally computes
avg on top of all hash values.
A published solution is not available for this question yet.
An enterprise software designer wants to leverage the best of cloud to minimize the number of
administrative overheads associated with her big data pipeline while also getting on-demand
scalability without sacrificing flexibility. What option should she choose to best serve these needs?
6406531967372.
Build the data pipeline using Spark, data storage on HDFS, and deploy both on
Hadoop using IaaS
6406531967373.
Build the data pipeline using Python run on Serverless where the data is stored
on cloud storage
6406531967374.
Build the data pipeline using Spark PaaS option on top of data stored on cloud
storage
6406531967375.
Build the data pipeline using MapReduce, data storage on HDFS, and deploy
both on Hadoop using IaaS
A published solution is not available for this question yet.
Consider an application that can scale from handling 1000 users to handling 100 million users by
simply making copies of itself, is able to seamlessly react to machines going offline by again
making new copies of itself in new machines, and otherwise leaves behind no trace of its
workings. Which of the following statements is true?
6406531967376.
This application has adopted cloud-native design
6406531967377.
This application cannot be called as cloud-native since it is not observable at all
times
6406531967378.
This application cannot be called as cloud-native since it is not manageable
easily
6406531967379.
This application cannot be called as cloud-native since it is not resilient
A published solution is not available for this question yet.
Would IRCTC's railway ticket booking application be suitable for a serverless implementation?
6406531967380.
No, because each use of the application by a prospective passenger for ticket
search, date choosing, ticket seat blocking and ticket purchase is together a high complexity
operation
6406531967381.
No, because each use of the application by a prospective passenger for ticket
search, date choosing, ticket seat blocking and ticket purchase will take a long time
6406531967382.
No, because a failed step in the use of the application can result in duplicate
payments
6406531967383.
No, because the number of ticket searches cannot be scaled by simply running
the application in multiple machines
6406531967384.
No, because multiple prospective passengers have to use on a single machine
(i.e. no embarrassingly parallel computation)
A published solution is not available for this question yet.
You are provided with a Spark program that picks out a list of suspicious transactions based on
the amount of the transaction being higher than a threshold and the geographic location of the
transaction matching a known set of suspected locations provided in a small file. The Spark
program is written to pull all the transactions from the Workers to the Driver and then apply the
logic of threshold filtering and geographic location matching. Which of the following changes will
you make to the program to improve performance?
6406531967393.
Use broadcast variables
6406531967394.
Hardcode threshold value as a filter condition in the Driver program
6406531967395.
Filter for transactions in the Workers before bringing to the Driver
6406531967396.
Hardcode threshold value as a filter condition in the Workers itself
A published solution is not available for this question yet.
Consider a file “data.bin” which is formatted as follows: every data record is in the form of pairs of
values of the form “column name,column value”, with each pair in its own line. You are asked to
write a data processing script using Python that scales with big data. Which of the following
represents your approach?
6406531967404.
Since data.bin is compliant with the RFC 4180, use PySpark’s read_csv() to read
the data as is.
6406531967405.
Rename the file data.bin to data.csv to make it compliant with RFC 4180 and
then use PySpark’s read_csv() to read the data
6406531967406.
The problem cannot be solved since the file cannot be converted to a valid
format for reading consistently without additional information
6406531967407.
Write PySpark code to read all lines in data.bin, use string split on “,” as
delimiter, and then collect all column names and corresponding values into a RDD for further
processing
A published solution is not available for this question yet.
Which of the following types of data sources can you read successfully without missing data using
a program that extracts once every day? If no data is missed, then all data in source will match
data extracted successfully.
6406531967426.
A MySQL table where all Creates are new rows and all Updates are in-place,
and these can happen anytime during the day.
6406531967427.
A Linux machine’s system.log file into which all processes append their events
6406531967428.
Share transactions stored as facts & share prices stored as SCD Type II
dimension tables in a PostgreSQL database
6406531967429.
A weather API that provides the temperature & rainfall readings for all its
weather stations in India for the specific time instant when being queried
6406531967430.
Google drive account that stores all historical census data for India
A published solution is not available for this question yet.
Which of the following statements is/ are true for Google Cloud Functions?
6406531967437.
Google Cloud functions work on stateless paradigm - one function invocation
doesn’t depend on the previous function invocations.
6406531967438.
Google cloud functions can be triggered by deletion of an object in GCS.
6406531967439.
Google Cloud Functions can automatically scale-up compute in case of high
load.
6406531967440.
A single execution of a google cloud function is timed indefinitely till the
operation finishes.
**Fin Forensics**
**Section Id :** 64065339864
**Section Number :** 8
**Section type :** Online
A published solution is not available for this question yet.
