Getting started with the built-in wide and deep algorithm

Console

Before you start your training job, you need to copy the data from our public Cloud Storage bucket to your Cloud Storage bucket.

Copy the sample data to your Cloud Storage bucket

1. First, download the training and testing data from our public Cloud Storage bucket.

   1. Navigate to our public Cloud Storage bucket:

      Get the sample data

   2. Download both test.csv and train.csv:

      1. Click the file name.

      2. From the Object details page, click Download. These files download to your local environment as ai-platform_census_algorithms_data_test.csv and ai-platform_census_algorithms_data_train.csv respectively.

2. Next, upload the training and testing data to your Cloud Storage bucket.

   1. Navigate to the Browser page for your Cloud Storage bucket. Select your project from the Select a project drop-down list, or open it in a new tab:

      Cloud Storage Browser page

   2. Click the name of the bucket you want to use, or create a new bucket if you do not have one. (If you create a new bucket, make sure it is a regional bucket, and select the same region where you're running the AI Platform training job.)

   3. (Optional) Click Create folder to create a folder for the files you upload. Enter a name for the folder (for example, "data") and click Create. Then, navigate to the new folder by clicking the folder name.

   4. Click Upload files to upload both the training and testing files, ai-platform_census_algorithms_data_train.csv and ai-platform_census_algorithms_data_test.csv to your bucket.

Now that the data is copied to your bucket, you can start a training job by selecting the type of algorithm you want to use.

Select your algorithm

1. Go to the AI Platform Jobs page in the Google Cloud Platform Console:

   AI Platform Jobs page

2. Click the New training job button. From the options that display below, click Built-in model training. The Create a new training job page displays.

3. The training job creation is divided into four steps. The first step is Training algorithm. Select Built-in wide and deep and click Next.

gcloud

Set up environment variables for your project ID, your Cloud Storage bucket, the Cloud Storage path to the training data, and your algorithm selection.

AI Platform built-in algorithms are in Docker containers hosted in Container Registry.

PROJECT_ID="[YOUR-PROJECT-ID]"
BUCKET_NAME="[YOUR-BUCKET-NAME]"
REGION="us-central1"
gcloud config set project $PROJECT_ID
gcloud config set compute/region $REGION

# Copy the training data into your Cloud Storage bucket, and set the path
# to your copy of the training data.
TRAINING_DATA_SOURCE="gs://cloud-samples-data/ai-platform/census/algorithms/data/train.csv"
TRAINING_DATA_PATH="gs://$BUCKET_NAME/algorithms-demo/data/train.csv"
gsutil cp $TRAINING_DATA_SOURCE $TRAINING_DATA_PATH

# Specify the Docker container URI specific to the algorithm.
IMAGE_URI="gcr.io/cloud-ml-algos/wide_deep_learner_cpu:latest"

Console

1. Leave Enable automatic data preprocessing checked.

2. For Training data path, click Browse. In the right panel, click the name of the bucket where you uploaded the training data, and navigate to your ai-platform_census_algorithms_data_train.csv file.

3. Leave the fields for Validation data and Test data at their default settings.

4. In Output directory, enter the path to your Cloud Storage bucket where you want AI Platform to store the outputs from your training job. You can fill in your Cloud Storage bucket path directly, or click the Browse button to select it.

   To keep things organized, create a new directory within your Cloud Storage bucket for this training job. You can do this within the Browse pane.

   Click Next.

5. For Model type, select Classification.

6. For Model type, select Classification.

7. Leave all other fields on their default settings, and click Next.

8. On the Job settings page:

   1. Enter a unique Job ID (such as "wide_deep_example").
   2. Enter an available region (such as "us-central1").
   3. Select "BASIC" for the scale tier.

   Click Done to submit the training job.

gcloud

1. Set up all the arguments for the training job and the algorithm, before using gcloud to submit the job:

   DATASET_NAME="census"
   ALGORITHM="wide_deep"
   MODEL_TYPE="classification"
   MODEL_NAME="${DATASET_NAME}_${ALGORITHM}_${MODEL_TYPE}"
   
   # Give a unique name to your training job.
   DATE="$(date '+%Y%m%d_%H%M%S')"
   JOB_ID="${MODEL_NAME}_${DATE}"
   
   # Make sure you have access to this Cloud Storage bucket.
   JOB_DIR="gs://${BUCKET_NAME}/algorithms_training/${MODEL_NAME}/${DATE}"
   

2. Submit the job:

   gcloud ai-platform jobs submit training $JOB_ID \
     --master-image-uri=$IMAGE_URI --scale-tier=BASIC --job-dir=$JOB_DIR \
     -- \
     --preprocess --model_type=$MODEL_TYPE --batch_size=250 --learning_rate=0.1 \
     --dnn_learning_rate=0.005 --dnn_dropout=0.1 --hidden_units=10,10,10 \
     --use_wide --embed_categories \
     --max_steps=1000 --training_data_path=$TRAINING_DATA_PATH
   

3. After the job is submitted successfully, you can view the logs using the following gcloud commands:

   gcloud ai-platform jobs describe $JOB_ID
   gcloud ai-platform jobs stream-logs $JOB_ID
   

Console

1. On the Jobs page, you can find a list of all your training jobs. Click the name of the training job you just submitted ("wide_deep_example" or the job name you used).

2. On the Job details page, you can view the general progress of your job, or click View logs for a more detailed view of its progress.

3. When the job is successful, the Deploy model button appears at the top. Click Deploy model.

4. Select Deploy as new model, and enter a model name, such as "wide_deep_model". Next, click Confirm.

5. On the Create version page, enter a version name, such as "v1", and leave all other fields at their default settings. Click Save.

gcloud

The training process with the built-in wide and deep algorithm produces a file, deployment_config.yaml, that makes it easier to deploy your model on AI Platform for predictions.

1. Copy the file to your local directory and view its contents:

   gsutil cp $JOB_DIR/model/deployment_config.yaml .
   cat deployment_config.yaml
   

   Your deployment_config.yaml file should appear similar to the following:

   deploymentUri: gs://[BUCKET_NAME]/algorithms_training/census_wide_deep_classification/model
   framework: TENSORFLOW
   labels:
     global_step: '1000'
     job_id: census_wide_deep_classification_20190227060114
     accuracy: '86'
   runtimeVersion: '1.12'
   

2. Create the model and version in AI Platform:

   MODEL_NAME="${DATASET_NAME}_${ALGORITHM}_${MODEL_TYPE}"
   gcloud ai-platform models create $MODEL_NAME --regions $REGION
   
   # Create a model and a version using the file above.
   VERSION_NAME="v_${DATE}"
   
   gcloud ai-platform versions create $VERSION_NAME \
     --model $MODEL_NAME \
     --config deployment_config.yaml
   

   The version takes a few minutes to create.