AWS Machine Learning - Specialty (MLS-c01 – 2020)

Based on the graph, we can observe that the decrease in the sum of squared errors (SSE) is significant until k=4, after which the decrease becomes less pronounced. This suggests that k=4 is a reasonable selection for the optimal choice of k, as it provides a good balance between minimizing the SSE and avoiding overfitting.

The correct answer is to use AWS KMS to encrypt the data on Amazon S3 and Amazon SageMaker, and redact the credit card numbers from the customer data with AWS Glue. This solution ensures that the data is encrypted both at rest and in transit, using AWS KMS for encryption. Additionally, AWS Glue can be used to remove or redact the credit card numbers from the dataset, further enhancing the security of the data.

The team should create an endpoint configuration with production variants for the two models with equal weights. This means that both models will be deployed and used simultaneously, with an equal amount of traffic being directed to each model. By doing this, the team can compare the long-term effectiveness of the two versions in reaching the related business outcome. This approach requires minimum management as the models are deployed and managed automatically by Amazon SageMaker.

The correct answer is to launch the SageMaker notebook instance within the VPC and create an S3 VPC endpoint for the notebook to access the data, and define a custom S3 bucket policy to only allow requests from the VPC to access the S3 bucket. This ensures that the data stored in Amazon S3 remains private and can only be accessed by the Specialist through the VPC and the designated notebook instance. The S3 VPC endpoint establishes a private connection between the VPC and S3, eliminating the need for internet access. The custom S3 bucket policy further restricts access to the bucket, ensuring the integrity of the data.

The Amazon SageMaker Linear Learner algorithm is most likely to provide the best results in this scenario. The task is to forecast air quality based on historical data, which is a regression problem. The Linear Learner algorithm is designed for regression tasks and can effectively learn patterns and make predictions based on the given time series data. Using a regressor predictor_type will allow the algorithm to accurately forecast the air quality in parts per million of contaminants for the next 2 days. The other options, such as k-Nearest-Neighbors and Random Cut Forest, may not be as suitable for this specific task.

The Specialist wants to determine the appropriate SageMaker Variant Invocations Per Instance setting for endpoint automatic scaling. They performed a load test and found that the peak requests per second (RPS) without service degradation is 20 RPS. Since the invocations per instance setting is measured on a per-minute basis, the Specialist intends to set the invocation safety factor to 0.5. To calculate the appropriate setting, we multiply the peak RPS by 60 (to convert it to requests per minute) and then divide it by the invocation safety factor (0.5). Therefore, the Specialist should set the SageMaker Variant Invocations Per Instance setting to 600.

The company should use Amazon Rekognition Video to extract metadata from the videos. Amazon Rekognition Video is a service that can analyze and extract meaningful metadata from videos, such as objects, scenes, and faces. By using this service, the company can automatically extract relevant information from the videos, which can then be used for creating a searchable video library. This approach would require minimal cost and limited management, as the extraction of metadata can be automated using Amazon Rekognition Video.

To initialize the model and re-train it with the custom data, the Machine Learning Specialist should use the option of initializing the model with pre-trained weights in all layers and replacing the last fully connected layer. This approach allows the Specialist to leverage the knowledge and features learned by the existing model trained on general objects. By replacing the last fully connected layer, the model can be adapted to the specific task of identifying the make and model of vehicles using the custom dataset.

The Machine Learning Specialist should use binary classification for this task because the goal is to predict whether or not a customer is likely to switch to a competitor in the next 30 days. Binary classification is the appropriate modeling technique for predicting a binary outcome, in this case, whether a customer will switch or not. Time-series prediction would be more suitable if the goal was to predict future usage patterns, while anomaly detection is used to identify unusual or abnormal behavior. Regression is used for predicting continuous numerical values, which is not the case here.

not-available-via-ai

The overfitting problem occurs when a model performs well on the training data but poorly on the testing data, indicating that the model is not generalized enough. Increasing the dropout rate at the flatten layer helps to regularize the model by randomly dropping out some of the neurons during training, preventing over-reliance on specific features and improving generalization. Therefore, increasing the dropout rate is the appropriate solution to address the overfitting issue in this scenario.

Storing the records in Amazon S3 in Parquet file format can improve query performance. Parquet is a columnar storage file format that is optimized for query performance. It uses compression techniques and column-level encoding to reduce the amount of data that needs to be read from disk during queries. This can significantly improve the query execution time, especially for large volumes of data like in this scenario where 1 TB of data is generated every minute.

The likelihood of flipping a coin "heads" can be described by the Binomial Distribution. This distribution is used when there are two possible outcomes (in this case, heads or tails) and each flip is independent. The Binomial Distribution calculates the probability of a certain number of successes (in this case, heads) in a fixed number of trials (the number of times the coin is flipped).

Data normalization is the process of scaling the data to a standard range. In this case, the high magnitude of some features can cause the neural network to give more importance to those features, leading to slower convergence during backpropagation. By normalizing the data, the features will be on a similar scale, allowing the neural network to learn more effectively and converge faster. This helps to prevent any one feature from dominating the cost function and ensures better convergence during backpropagation.

The Specialist should download the TensorFlow Docker container used in Amazon SageMaker from GitHub to their local environment and use the Amazon SageMaker Python SDK to test the code. This approach allows the Specialist to have a local copy of the SageMaker environment, including the necessary dependencies and libraries, without requiring an internet connection. They can continue working on the project and test the code using the SageMaker Python SDK, ensuring compatibility and consistency with the SageMaker environment.

Amazon Kinesis Data Firehose is the correct answer because it is a fully managed service that can ingest streaming data and store it in Apache Parquet files. It can automatically convert the incoming data to the desired format and deliver it to various destinations, including Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service. This makes it suitable for exploration and analysis of the data.

To create a notebook, the ML Specialist should SSH into the Deep Learning AMI instance with port forwarding at port 8888. They should then start a Jupyter notebook application, which will provide them with the notebook interface. From there, they can create a new ipython notebook.

The company should use AWS Glue for data preprocessing because it is a serverless data integration service that can efficiently process large structured data. Saving the data in Amazon S3 in Parquet format is recommended because Parquet is a columnar storage format that provides better compression and query performance compared to CSV format. This solution will help minimize the cost for data storage and compute while integrating with the existing ML product that uses Amazon EMR with Spark.

The company should use Amazon SageMaker DeepAR algorithm to build the model. DeepAR is a time series forecasting algorithm that is specifically designed for predicting future values based on historical data. In this case, the algorithm can be trained on the consumption metrics of similar factory machines to predict when a machine part is about to stop working. The algorithm's ability to handle time series data and capture temporal dependencies makes it suitable for this task.

The most suitable predictive model that can be deployed into production is to add labels over time to indicate which engine faults occur at what time in the future, turning this into a supervised learning problem. Then, a recurrent neural network (RNN) can be used to train the model to recognize when an engine might need maintenance for a certain fault. This approach allows the model to learn patterns and correlations in the data over time, making it effective for predicting engine problems based on the collected sensor readings.

The Specialist needs to perform three operations to correctly sanitize and prepare the data in a repeatable manner. Firstly, normalizing all words by making the sentence lowercase ensures that the capitalization of words does not affect the analysis. Secondly, removing stop words using an English stopword dictionary helps to eliminate commonly used words that do not carry much meaning. Lastly, tokenizing the sentence into words breaks it down into individual units, which is necessary for further processing and analysis.

The correct solution is to train an Amazon SageMaker DeepAR algorithm to forecast demand for the new product. The DeepAR algorithm is specifically designed for time series forecasting, which is the case for predicting demand for consumer goods. It can handle multiple related time series and capture complex patterns and dependencies in the data. Therefore, it is the most suitable solution for forecasting demand for the new product based on the sales history of the existing products.

A rule-based solution should be used instead of ML because the company already has specific criteria (previous user feedback) to identify IP addresses connected to spam and phishing websites. ML models require training data, which may not be readily available in this case. By using a rule-based solution, the company can set predefined rules based on the feedback to flag the IP addresses without the need for ML algorithms.

The residuals are the differences between the predicted values and the actual values in a regression model. By tracking the residuals on a chart, the Machine Learning specialist can identify any consistent pattern of overestimation. If the residuals consistently have positive values, it indicates that the model is consistently overestimating the outcome. Therefore, tracking the residuals can help in identifying the root cause of the underperformance of the regression model.

Residual plots can be used to determine whether a regression model is overestimating or underestimating the target value. Residual plots show the difference between the predicted and actual values of the target variable. If the majority of the residuals are positive, it indicates that the model is overestimating the target. Conversely, if the majority of the residuals are negative, it indicates that the model is underestimating the target. Therefore, by analyzing the pattern of residuals in the plot, the Machine Learning Specialist can understand whether the model is more frequently overestimating or underestimating the target.

SageMaker Distributed Training is a feature within Amazon SageMaker that enables large-scale distributed training of machine learning models across multiple instances. This advanced capability is particularly useful for handling large datasets and complex model training scenarios, making it an essential tool for scaling machine learning workflows in AWS.

The Data Scientist should make sure that the AWS Identity and Access Management (IAM) role used for Amazon S3 access has the necessary permissions to encrypt and decrypt the data with the AWS KMS key. This is because the data is encrypted at rest and stored in an Amazon S3 bucket using customer-managed encryption with AWS KMS. Without the proper permissions, the Data Scientist will not be able to access and download the encrypted data for use in the Amazon SageMaker training job, resulting in the "Data download failed" error.

The ML Specialist should use a Poisson distribution for this variable because it represents the number of events (minutes of waiting) occurring in a fixed interval (bus cycle every 10 minutes) with a known average rate (mean of 3 minutes). The Poisson distribution is commonly used for modeling the number of events in a fixed interval, making it the appropriate choice in this scenario.

The goal is to predict the sales levels of each potential next product and select the one with the highest predicted purchase rate. This is a regression problem because the target variable (sales levels) is a continuous variable. Linear regression is an appropriate machine learning approach for regression problems as it aims to model the relationship between the independent variables (such as product features, marketing efforts, etc.) and the dependent variable (sales levels) through a linear equation. By using linear regression, the company can make predictions on the sales levels of each potential product and choose the one with the highest predicted sales.

The Specialist needs to build the Docker container to be NVIDIA-Docker compatible. This is because the Specialist is using Amazon EC2 P3 instances, which have NVIDIA GPUs. By making the Docker container NVIDIA-Docker compatible, the Specialist ensures that the container can properly leverage the NVIDIA GPUs for training the model. This compatibility allows the container to access the necessary NVIDIA drivers and libraries for GPU acceleration.

The Specialist should use a Classification model to accomplish this task. Classification models are used to predict the category or class that an observation belongs to based on its features. In this case, the Specialist wants to group customers into categories of churn or no churn, which is a binary classification problem. By training a classification model on the labeled data, the Specialist can predict whether a customer will churn or not based on their features.

An XGBoost model with the objective parameter set to multi: softmax should be used for categorizing new products using the provided dataset for training. XGBoost is a powerful machine learning algorithm that can handle multi-class classification problems effectively. By setting the objective parameter to multi: softmax, the model will be trained to output probabilities for each category, allowing it to classify new products accurately. This model is suitable for the given dataset as it has multiple features and six different categories to classify the products into.

Creating a hyperparameter tuning job and setting the accuracy as an objective metric is the most repeatable and requires the least amount of effort to achieve the desired results. This approach automates the process of searching for the best hyperparameters by iteratively training and evaluating models with different hyperparameter configurations. By setting the accuracy as the objective metric, the tuning job will optimize the hyperparameters to maximize the model's accuracy. This method eliminates the need for manual trial and error and ensures a systematic and efficient approach to improving the model's accuracy.

To scale the training solution for future demand, the Machine Learning Specialist should change the TensorFlow code to implement a Horovod distributed framework supported by Amazon SageMaker. This will allow the training to be parallelized across multiple machines, enabling faster training and accommodating the anticipated increase in training data size. By using Horovod and Amazon SageMaker, the Specialist can achieve the business goals without making significant changes to the coding effort and infrastructure.

The best metric to score the model in this scenario is the Area Under the ROC Curve (AUC). AUC is a suitable metric for evaluating models on unbalanced datasets because it measures the model's ability to distinguish between the positive and negative classes. In this case, the positive class represents the fraudulent transactions, and the negative class represents the valid transactions. AUC takes into account both true positive rate (sensitivity) and false positive rate (1-specificity), which is important when identifying both fraudulent and valid transactions is equally important.

The team should use Amazon SageMaker Object2Vec to create a vector and then use the Amazon SageMaker built-in Sequence to Sequence model (Seq2Seq). This approach allows them to create a vector representation of the news articles using Object2Vec, which can capture the semantic meaning of the text. Then, they can use the Seq2Seq model to train a machine learning model for each non-English language. The built-in Seq2Seq model in Amazon SageMaker simplifies the management of the models as it provides pre-built functionality for sequence-to-sequence tasks, reducing the need for additional management efforts.

The company should use AWS Glue to transform files because it is a fully managed extract, transform, and load (ETL) service that makes it easy to prepare and load data for analytics. AWS Glue supports Apache Spark and can be used to transform the geographical data columns and add additional features for the k-nearest neighbors algorithm. Using Amazon S3 as the destination for the transformed data is a good choice because it is a scalable and cost-effective storage solution. This solution meets the company's requirements of minimal changes to the current pipeline and minimal management.

The Specialist should frame this business problem as binary classification because the goal is to predict whether a transaction is fraudulent or not. Binary classification is used when there are only two possible outcomes, in this case, fraudulent or not fraudulent.

The best way to quickly and efficiently design and build a service that gives insight into social media sentiment for the marketing team is to use Amazon Translate, Amazon Comprehend, Amazon Kinesis, Amazon Athena, and Amazon QuickSight. These services provide a comprehensive solution for natural language processing (NLP) and data analysis. Amazon Translate can be used to translate social media content into different languages, Amazon Comprehend can be used for sentiment analysis, Amazon Kinesis can be used for real-time data streaming, Amazon Athena can be used for querying and analyzing the data, and Amazon QuickSight can be used for visualizing the insights on a dashboard. This combination of services enables the marketing team to understand the sentiment of their products across different languages and make informed decisions for the roll out of a new product.

To prepare the data for the LSTM, the team should vectorize the sentences by transforming them into numerical sequences. Additionally, padding should be applied to ensure that all sequences have the same length. This is important because LSTMs require fixed-length input. By vectorizing and padding the sentences, the data can be effectively processed by the LSTM model for classification.

Kinesis Firehose is the correct answer for this question. Kinesis Firehose is a fully managed service that automatically scales to handle the transfer of data, such as Apache web logs, syslogs, text, and videos, from various sources to Amazon S3. It requires minimum transformation, making it suitable for the given scenario where the analytics company wants to transfer their web logs, syslogs, text, and videos to Amazon S3 without extensive data manipulation.

This approach meets the requirements because it utilizes AWS Glue, which is a fully managed extract, transform, and load (ETL) service. The AWS Glue crawler is used to automatically discover and catalog metadata about the data in the S3 data lake. An AWS Lambda function is used to trigger the AWS Glue ETL job, which allows for event-driven ETL pipelines. The AWS Glue Data Catalog is used to search and discover metadata, providing a quick and easy way to understand the data lake's metadata. This approach also aligns with the requirement of supporting querying old and new data through Amazon Athena and Amazon Redshift Spectrum.

When using the AWS SDK for Python (Boto 3) library, the method to call for inferences from a model deployed to Amazon SageMaker Hosting Services is "invoke_endpoint". This method is used to send a request to the endpoint and receive the inference response. It allows you to pass input data to the model and obtain the corresponding predictions or inferences.

Performing data augmentation on the training data can help resolve the issue of the model not accurately inferring on unseen test images. Data augmentation involves creating new training data by applying various transformations to the existing training images, such as rotating, scaling, or flipping them. This can help the model generalize better and improve its performance on unseen data.

Adding L2 regularization to the model can also be helpful in resolving this issue. L2 regularization adds a penalty term to the loss function during training, which discourages the model from overfitting the training data. By reducing overfitting, the model is more likely to perform well on unseen test images.

The correct answer is to set the predictor_type hyperparameter to binary_classifier. This is because a binary classifier is designed to produce discrete results by classifying data into two categories, such as yes or no, true or false. In this case, the goal is to determine whether a customer should receive an email from the current campaign or not, which is a binary classification problem. By setting the predictor_type hyperparameter to binary_classifier, the SageMaker Linear Learner algorithm will be trained to produce discrete results for this specific task.

The recall of a model is the ratio of the true positive predictions to the sum of true positive and false negative predictions. In this case, the recall is 0.92, which means that the model has a high ability to correctly identify positive instances.

The precision of a model is the ratio of the true positive predictions to the sum of true positive and false positive predictions. In this case, the precision is 0.8, which means that the model has a moderate ability to avoid false positive predictions.

Therefore, the correct answer is Recall = 0.92 Precision = 0.8.

The simplest approach with the least risk to deploy the model and roll it back, if needed, is to update the existing SageMaker endpoint to use a new configuration that is weighted to send 5% of the traffic to the new variant. This approach allows for gradual testing and monitoring of the new model's performance with a small percentage of traffic, minimizing the impact on customers. If the model does not perform as expected, traffic can be reverted to the last version by resetting the weights.

The Specialist should download word embeddings pre-trained on a large corpus to meet the requirements. Word embeddings are dense vector representations of words that capture semantic relationships between words. By using pre-trained word embeddings, which have been trained on a large corpus of text data, the Specialist can leverage the learned semantic relationships to provide word features for the downstream nearest neighbor model. This will enable the widget to display words used in similar contexts based on the semantic similarities captured by the word embeddings.

There are 2 sentences, 8 unique unigrams, and 8 unique bigrams, so the result would be (2,16). The
phrases are “Please call the number below” and “Please do not call us.” Each word individually (unigram) is
“Please,” “call,” ”the,” ”number,” “below,” “do,” “not,” and “us.” The unique bigrams are “Please call,” “call the,” ”the
number,” “number below,” “Please do,” “do not,” “not call,” and “call us.”

The data engineer should deploy Amazon S3 Standard for the processed data that is within one year of processing. After one year, they should use Amazon S3 Glacier for the processed data. Additionally, they should use Amazon S3 Glacier Deep Archive for all raw data. This solution ensures that the processed data is highly available for at least one year, allowing data analysts and data scientists to run analytics and ML workloads on the most recent data. The use of Amazon S3 Glacier Deep Archive for raw data ensures compliance and accessibility within 24 hours when needed.

The Data Scientist can ask the social media handling team to review each post using Amazon SageMaker GroundTruth and provide the label. This approach involves manual review and labeling of each post by the team, ensuring accurate target labels for training. Additionally, the Data Scientist can use Amazon Mechanical Turk to publish Human Intelligence Tasks that ask Turk workers to label the posts. This crowdsourcing approach allows for a larger pool of workers to label the posts, increasing efficiency and scalability in generating valid target label data.

The Data Scientist believes that the poor validation accuracy may be due to a rich vocabulary and low average frequency of words in the dataset. In order to improve the accuracy, they should use Scikit-learn term frequency-inverse document frequency (TF-IDF) vectorizers. TF-IDF is a technique that assigns weights to words based on their frequency in a document and their rarity in the entire dataset. By using TF-IDF vectorizers, the Data Scientist can give more importance to the words that are both frequent in a document and rare in the dataset, which can help improve the accuracy of the sentiment analysis application.

The Specialist should use AWS Glue as the data catalog to manage the metadata of the data lake. They should use Amazon Kinesis Data Streams and Amazon Kinesis Data Analytics for real-time data insights, allowing them to process and analyze streaming data in real-time. They should also use Amazon Kinesis Data Firehose to deliver the clickstream data to Amazon ES for clickstream analytics. Lastly, they should use Amazon EMR to generate personalized product recommendations by processing and analyzing the data in the data lake.

Setting the input mode for the training job to Pipe allows the data to be streamed directly from Amazon S3 to the training instances, which can significantly reduce the time it takes for the training job to launch. This can help address the observed issue of the unacceptable length of time before the training job launches.

The company should use the Build Your Own Container (BYOC) Amazon SageMaker option. This option allows the company to create a new Docker container with the existing code and register it in Amazon Elastic Container Registry (ECR). By using this option, the company can port their training and inference code written in R to Amazon SageMaker with minimal changes. They can then run the training and inference jobs using this container, ensuring a seamless transition to the new ML pipeline.

To address the issue of mispronounced company acronyms in future documents, a Machine Learning Specialist should create an appropriate pronunciation lexicon. This lexicon will contain the correct pronunciation of the acronyms, allowing Amazon Polly to accurately pronounce them in the speech output. By creating a pronunciation lexicon, the specialist can ensure that the acronyms are pronounced correctly in all future automated company announcements.

The financial organization can use AWS Batch to automate the batch data preprocessing and ML training aspects of their pipeline. AWS Batch is a fully managed service that allows users to run batch computing workloads on the AWS Cloud. It takes care of the underlying infrastructure and automatically scales resources to meet the demands of the workload. With AWS Batch, the organization can schedule their data preprocessing and ML training tasks to run automatically every 24 hours, ensuring a streamlined and automated process.

XGBoost is a popular machine learning algorithm that is commonly used for classification problems, such as fraud detection. It is known for its ability to handle large datasets and its effectiveness in capturing complex relationships between variables. Therefore, it would be a suitable choice for the bank's Machine Learning team to use XGBoost for modeling the credit card fraud detection problem.

To prepare the data for use in an Amazon SageMaker training job, the ML Engineer should first change the target label to the first column. Then, the data should be split into training, validation, and test sets. Finally, the datasets should be uploaded to Amazon S3.

It is most likely that the loss function is very curvy and has multiple local minima where the training is
getting stuck. Decreasing the batch size would help the data scientist stochastically get out of the local minima
saddles. Decreasing the learning rate would prevent overshooting the global loss function minimum

The company should launch the notebook instances in a private subnet and access the data through an S3 VPC endpoint. This setup ensures that the data is not transmitted across the internet, as required by the compliance policy. By using a private subnet, the instances are not accessible from the public internet. The S3 VPC endpoint allows the instances to securely access the S3 bucket within the VPC, without the need for internet connectivity. This ensures that the data remains within the company's network and complies with the compliance policy.

Random Search and Bayesian optimization are two techniques that can be used to tune the hyperparameters of a machine learning model on Amazon SageMaker to improve the model's F1 score. Random Search involves randomly selecting combinations of hyperparameters from a predefined search space and evaluating their performance. Bayesian optimization, on the other hand, uses a probabilistic model to find the optimal set of hyperparameters by iteratively exploring the search space based on previous evaluations. Both techniques can help identify the best hyperparameter values that maximize the F1 score.

The Amazon SageMaker SDK uses the HyperparameterTuner() API to create and interact with the Amazon SageMaker hyperparameter tuning jobs. This API allows the data scientist to automate the tuning process for their random forest model, optimizing the hyperparameters to improve the model's performance.

To avoid loading the entire large dataset onto the limited storage of the SageMaker notebook instance, the Machine Learning Specialist should load a smaller subset of the data into the notebook and train locally. This allows them to confirm that the training code is executing correctly and the model parameters are reasonable. Once this is verified, they can initiate a SageMaker training job using the full dataset from the S3 bucket using Pipe input mode. This approach allows the Specialist to use all the data for training without exceeding the storage limitations of the notebook instance.

The correct answer is to use AWS Lambda to trigger an AWS Step Functions workflow to wait for dataset uploads to complete in Amazon S3. This configuration meets the requirements as it ensures that the workflow starts as soon as data is uploaded to S3. AWS Glue is used to join the datasets, and the results are stored in S3. An Amazon CloudWatch alarm is set up to send an SNS notification to the Administrator in case of a failure. This configuration provides an efficient and scalable solution for the daily ETL workflow.

The company should use T-SNE and PCA to reduce the dimensionality of the data. T-SNE is a technique used for visualizing high-dimensional data by reducing it to a lower-dimensional space while preserving the structure of the data. It is particularly useful for visualizing clusters or patterns in the data. PCA, on the other hand, is a technique used to reduce the dimensionality of data by finding a new set of variables (principal components) that capture the most important information in the original data. By visualizing the data using matplotlib, the company can gain insights into the distribution and relationships between variables.

Amazon Kinesis and AWS Data Pipeline are both services that can feed data to MapReduce jobs. Amazon Kinesis is a real-time streaming service that can continuously collect and process large streams of data in near-real time. AWS Data Pipeline is a web service for orchestrating and automating the movement and transformation of data between different AWS services and on-premises data sources. Both of these services can be used to feed data to the MapReduce jobs for processing and analysis.

The following calculations are required:

TP = True Positive
FP = False Positive
FN = False Negative
TN = True Negative
FN = False Negative

Recall = TP / (TP + FN)

False Positive Rate (FPR) = FP / (FP + TN)

Cost = 5 * FP + FN

Options C and D have a recall greater than 80% and an FPR less than 10%, but D is the most cost effective.

The ML Engineer should use a box plot to visualize the feature in order to check for outliers. A box plot provides information about the minimum, first quartile, median, third quartile, and maximum values of a dataset, allowing the Engineer to identify any extreme values that may be considered outliers. Additionally, the Engineer should use a histogram to visualize the overall distribution and skewness of the feature. A histogram provides a visual representation of the frequency distribution of the feature, allowing the Engineer to observe the shape of the distribution and identify any skewness.

The Specialist should consider increasing regularization to prevent overfitting and improve generalization on the test data. Increasing dropout can also help reduce overfitting by randomly dropping out neurons during training. Decreasing feature combinations can help simplify the model and reduce the risk of overfitting.

not-available-via-ai

In this case, a full review summary usually contains the most descriptive phrases of the entire review and is
a valid stand-in for the missing full review text field.

To update the endpoint without any downtime and minimal code changes, the team should first de-register the endpoint as a scalable target. Then, they should update the endpoint using a new endpoint configuration that includes the latest model Amazon S3 path. Finally, they should register the endpoint as a scalable target again. This process ensures that the endpoint is temporarily taken out of the scaling policy, allowing the team to update it with the latest model while minimizing disruption to the customers.

The company can use Amazon Lex to parse the utterances and intent of customer comments, and Amazon Polly to generate personalized replies to the customers. Amazon Lex is a service for building conversational interfaces using voice and text, which can understand and interpret customer inputs. Amazon Polly is a service that turns text into lifelike speech, allowing the chatbot to respond to customers in a natural and personalized manner. This combination of services allows the company to create intelligent chatbots that can provide personalized experiences without requiring a lot of ongoing management.

Neural networks have been widely adopted in the last decade due to several factors. Firstly, the development of efficient algorithms has made it easier to train and optimize neural networks, leading to improved performance. Additionally, the availability of cheaper GPUs (Graphics Processing Units) has significantly accelerated the computational power required for neural network training and inference. Lastly, there has been an exponential increase in the amount of data collected in various fields, allowing neural networks to learn from larger and more diverse datasets, leading to better performance and generalization.

The Specialist should transform the dataset into the Recordio protobuf format. This format is optimized for high-performance, efficient data storage and retrieval, which can improve the speed of training on SageMaker.

Using log-scaled hyperparameters allows for a more efficient search of the hyperparameter space. Log scaling reduces the range of values that need to be searched, making the optimization process faster. By taking the logarithm of the hyperparameters, the search space is compressed, allowing the algorithm to explore a wider range of values in a shorter amount of time. This helps in finding the optimal combination of hyperparameters more quickly and improving the overall optimization process.

The solution that requires the least coding effort is to run a daily Amazon EMR workflow to generate precision-recall data and save the results in Amazon S3. Then, the arrays can be visualized in Amazon QuickSight and published in a dashboard shared with the Business team. This solution minimizes coding effort by utilizing existing services and tools provided by Amazon, rather than building a custom solution from scratch.

To build an Alexa-type device that converses with customers using speech, the appropriate Amazon services to use would be Amazon Transcribe, Amazon Lex, and Amazon Polly. Amazon Transcribe is used to convert speech to text, Amazon Lex is used for natural language understanding and conversation management, and Amazon Polly is used for text-to-speech conversion. This combination of services allows for the device to understand and respond to customer queries using speech.

The confusion matrix is used to evaluate the performance of a classification model. It consists of four metrics: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). In this case, the question asks for the percent of positive records that were classified correctly, which is the definition of recall. Recall measures the ability of a model to correctly identify positive records out of all actual positive records. Therefore, the correct answer is 80%; Recall.

The Machine Learning team should use AWS CloudTrail to log Amazon SageMaker API calls to Amazon S3. This will allow the auditors to view the log activity report and ensure there are no unauthorized API calls. Additionally, the team should add code to push a custom metric to Amazon CloudWatch to monitor the model's performance. Finally, they should create an alarm in CloudWatch with Amazon SNS to receive a notification when the model is overfitting. This solution addresses the requirements with the least amount of code and fewest steps.

A SageMaker Endpoint in AWS is used for real-time inference, enabling predictions on new data by deploying an already trained machine learning model. It is not used for storing training data, batch processing, or hyperparameter tuning. Training data storage and batch processing are handled by other services, while hyperparameter tuning is managed through Hyperparameter Tuning Jobs. The Endpoint focuses on providing real-time predictions once a model is trained.

The option to insert an Amazon Kinesis Data Analytics stream downstream of the Kinesis Data Firehose stream that transforms raw record attributes into simple transformed values using SQL will require the least amount of development effort. This option leverages the existing data ingestion pipeline and adds a simple transformation step using SQL, which can be easily implemented. It eliminates the need for additional infrastructure setup like deploying EMR clusters or spinning up EC2 instances, making it a more efficient and cost-effective solution.

https://docs.aws.amazon.com/kms/latest/developerguide/control-access-overview.html#managing-access

Since Example Corp has sequential sales data from the past 15 years, it would be appropriate to use a sequential split for the training and evaluation datasets. This means that the data should be split in a way that maintains the order of the sales data. This will allow the model to learn from the past patterns and trends in the data, which is crucial for predicting future sales accurately.

The Data Scientist can use Adam, Adagrad, and RMSProp as optimizers to improve training performance. These optimizers are known for their ability to converge faster to the optimal solution compared to stochastic gradient descent (SGD). Adam combines the advantages of both AdaGrad and RMSProp, making it a popular choice. Adagrad adapts the learning rate for each parameter individually, which can be beneficial in dealing with sparse data. RMSProp also adapts the learning rate, but it uses a moving average of squared gradients to adjust the learning rate.

The Data Scientist should use the architecture described in option 2. This option suggests writing the raw data to Amazon S3 and using AWS Glue ETL job to perform the ETL processing. By writing the ETL job in PySpark, the existing logic can be leveraged. A new AWS Glue trigger can be created to trigger the ETL job based on the existing schedule. The output target of the ETL job can be configured to write to a "processed" location in Amazon S3, which is accessible for downstream use. This architecture meets all the given requirements, including combining multiple data sources, reusing existing PySpark logic, running on the existing schedule, and minimizing the number of managed servers.