The Amazon Web Services (AWS) Integration allows Virtana Platform to collect performance data at regular intervals from AWS for analysis in Cloud Cost Management (CCM). You can create a single or multiple integrations, depending on how many AWS instances you want to analyze with CCM.
There are 3 methods you can choose from to set up your AWS integration. But regardless of the method used, you must have permissions to perform administrative actions in both the AWS Console and in Virtana Platform Settings.
All setup methods also require you to move between the two UIs, to copy values from AWS and enter them into the Virtana integration setup form.
Note
In all AWS configuration tasks, the steps performed in AWS were accurate at the time the tasks were written. If the AWS interface changes, some steps might be different than described.
Tip
Ensure the AWS Security Token Service (STS) is active for all regions in which you have resources.
If STS is inactive, you will receive an error, "Invalid IAM role was rejected", when configuring the integration.
Steps to check the status of each region, as well as to make a region active, can be found in AWS documentation under the section titled "Activating and deactivating AWS STS in an AWS region".
Next Step
Before proceeding, ensure you have the proper roles and permissions.
The person who is performing the integration must have access to the AWS instance being added to Virtana Platform. During the setup process, Cloud Cost Management (CCM) must be given access to the AWS CloudWatch metrics.
Before starting the setup process, ensure you have the following roles to perform the necessary tasks on the AWS instance you are configuring with Virtana Platform:
-
Administrator role in AWS with permissions to enable Cost Explorer
-
Administrator role in AWS with permissions to create a Cost & Usage Report
-
Administrator role in AWS with permissions to create in-line policies
-
Administrator role in AWS with permissions to create roles and users
-
Administrator role in Virtana Platform with permissions to add an integration
Next Step
You must select the setup method for the AWS integration.
AWS strongly recommends using IAM roles for installation. However, access keys are also supported for configurations that require keys, such as AWS GovCloud environments.
You need to determine which setup method to use. Virtana Platform supports two methods for configuring the integration with IAM roles. You can use a CloudFormation Template, which automates most of the steps. You can also use the manual IAM role method.
Review the AWS Security Best Practices for more information.
You must select one of the following three methods to set up an AWS Integration in Virtana Platform:
-
CloudFormation Template for creating an IAM Role
Recommended method, as it's the simplest, quickest, and least error-prone.
-
Manual method for creating an IAM Role
More time-consuming and error-prone than the template method, but allows for more control.
-
Access Keys
Required for GovCloud accounts.
After you determine which setup method to use, you need to perform some tasks in AWS. You must also collect information from the AWS Management Console and provide those values in the integration setup form in Virtana Platform.
Next Steps
In AWS, enable Cost Explorer and then create a Cost & Usage Report (CUR).
AWS Cost Explorer must be enabled if you want Cloud Cost Management (CCM) to provide Bill Analysis reports. Regardless of the integration installation method used, Cost Explorer must be enabled from the management account, even if set up on a sub-account.
IAM Roles set up with the management account allow CCM to present reports spanning all of your accounts. IAM Roles set up with a sub-account only report cost for that one account.
-
Log in to your AWS management account.
-
Navigate to Cost Explorer.
-
Click Enable Cost Explorer.
The API is now available for use but has no data. You can request up to a year of cost billing data from AWS.
Next Step
Create a Cost & Usage Report in AWS
You must configure a Cost & Usage Report (CUR) in AWS prior to configuring the AWS integration. This is required for all AWS integration configurations if you want bill analysis reports.
CURs publish your AWS billing reports once a day in CSV format to an S3 bucket that you own. Virtana Platform's Cloud Cost Management (CCM) uses these reports to analyze your resource costs and right-sizing needs. (Cost & Usage Reports replaced Detailed Billing in AWS.)
About This Task
If you already have a Cost & Usage Report that is properly configured with hourly billing, ZIP or GZIP compression, and CSV file format for billing, then you can skip this task. However, if the CUR is not properly configured, CCM cannot interact correctly with AWS.
Steps
-
Log in to your AWS Console for your management account.
-
Navigate to Billing > Cost & Usage Reports.
-
Click Create report.
The Report Content page displays.
-
Name the report.
Example: HourlyCSVWithResourceIDs
-
Enable the Include resource IDs checkbox and click Next.
-
Click Configure to choose (or create) an S3 bucket to store your files and click Next.
-
Check "I have confirmed that this policy is correct" in the Verify Policy popup and click Save.
The Delivery Options page displays.
-
Provide a Report path prefix.
Example: CostAndUsageReports
Do not include any leading or trailing forward slashes. Doing so may distort the file hierarchy output by AWS.
Tip
Existing report path prefixes can be found by accessing the AWS Cost & Usage Reports and clicking a report name.
-
Make a note of the Report path prefix as you will need it when configuring AWS in Virtana Platform.
-
Select Hourly under Time granularity.
-
Select your preferred Report versioning method.
Overwriting the existing report might save on your storage costs in the future.
Leave all data integration options unchecked.
-
Select ZIP or GZIP for Compression type. and then click Next.
-
Review your configurations and then select Review and Complete to create the Cost and Usage Report.
Tip
It can take up to 24 hours for data to populate in the S3 bucket. After that, AWS updates the data at least once a day.
Next Steps
Configure AWS using the CloudFormation Template method, the manual method, or the access key method.
The CloudWatch Agent enables Virtana Platform to collect additional EC2 metrics, such as memory utilization, from AWS. Reports display cost vs. CPU utilization by default. You must install the agent on each AWS instance on which you want to view cost vs. memory utilization in Cloud Cost Management (CCM).
Tip
The CloudWatch Agent configuration below adds a single memory metric to all instances on which it is installed. This incurs an additional charge to your CloudWatch bill. See AWS CloudWatch pricing for more information.
Options for Configuring the CloudWatch Agent
Installing the CloudWatch agent can be done in a variety of ways, but each method requires the use of Virtana Platform's unique agent configuration file.
AWS offers 3 ways to install the CloudWatch Agent:
Complete the following tasks to install the agent on Linux and Windows instances using the CLI method and the AWS console.
-
SSH into your Linux instance.
-
Run one of the following to download the agent, depending on your Linux distribution:
Red Hat: wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
Debian: wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
-
Run one of the following to install the agent:
Red Hat: sudo rpm -i amazon-cloudwatch-agent.rpm
Debian: sudo dpkg -i -E ./amazon-cloudwatch-agent.deb
-
Navigate to the bin directory of the agent:
cd /opt/aws/amazon-cloudwatch-agent/bin
-
Create a file named
config.json
, place the Linux Agent Config File contents in it, and save the file.See below for JSON config file content.
-
Create an IAM role and attach it to the instance.
Follow the AWS links above for instructions.
The IAM role provides permissions for reading information from the instance and writing it to CloudWatch.
-
Run the following command to initialize the agent configuration:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:config.json -s
-
Start the agent:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a start
-
Verify the agent is running:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
-
A new metric cwagent.mem_used_percent is now available in AWS CloudWatch on the respective EC2 instance and can be accessed by CCM. The Memory Utilization metric in CCM will display an updated value the next time the cost reports run.
Example 1. Linux Agent Config File
{
"agent": {
"metrics_collection_interval": 60,
"run_as_user": "root"
},
"metrics": {
"append_dimensions": {
"InstanceId": "${aws:InstanceId}"
},
"metrics_collected": {
"collectd": {
"metrics_aggregation_interval": 60
},
"mem": {
"measurement": [
"mem_used_percent"
],
"metrics_collection_interval": 60
},
"statsd": {
"metrics_aggregation_interval": 60,
"metrics_collection_interval": 60,
"service_address": ":8125"
}
}
}
}
-
Log on to your Windows instance.
-
Download the following file:
https://s3.amazonaws.com/amazoncloudwatch-agent/windows/amd64/latest/amazon-cloudwatch-agent.msi
The file should immediately download.
-
Open the command prompt, navigate to the directory containing the downloaded file, and enter the following to install the agent:
msiexec /i amazon-cloudwatch-agent.msi
-
Open a text editor, create a new file, and place the Windows Agent Config File contents in it.
-
Save the file with the name amazon-cloudwatch-agent.json to the following directory:
C:\ProgramData\Amazon\AmazonCloudWatchAgent
-
Open Windows PowerShell and run the following command to fetch the configuration and start the agent:
& "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -c file:"C:\ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json" -s
-
Verify the agent is running:
& $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status
-
If necessary, start the agent using this command:
& $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a start
A new metric cwagent.memory % committed bytes in use is now available in AWS CloudWatch on the respective EC2 instance and can be accessed by CCM. The Memory Utilization metric will display an updated value the next time the CCM reports run.
Example 2. Windows Agent Config File
{
"agent": {
"metrics_collection_interval": 60,
"logfile": "c:\\ProgramData\\Amazon\\AmazonCloudWatchAgent\\Logs\\amazon-cloudwatch-agent.log"
},
"metrics": {
"metrics_collected": {
"Memory": {
"measurement": [
"% Committed Bytes In Use"
],
"metrics_collection_interval": 60
}
},
"append_dimensions": {
"InstanceId": "${aws:InstanceId}"
}
}
}
You can verify that the memory metric is being collected in AWS to ensure CloudWatch is properly configured.
-
Navigate to CloudWatch and in the left pane select Metrics > All Metrics.
-
On the Browse tab, under Metrics, locate Custom Namespaces and click the card labeled CWAgent.
New cards display with information for the CWAgent.
-
Click the card labeled Instanceid, objectname.
You should see a list of instances producing Memory Usage Metrics.
Cost vs Utilization (CvU) reports are updated daily in CCM, so the day after your AWS configuration is completed you should see Memory Utilization Metrics on the CvU report
If you receive right-sizing or cost vs utilization reports, you can verify in those reports that Memory Utilization values are displayed. The Rigjht Sizing report updates every Saturday and the Cost vs Utilization (CvU) report updates daily.
If you do not receive reports, you can verify the metric in the Cloud Cost Management (CCM) UI.
-
Navigate to the Cost Saving Opportunities page in CCM and click the Right Sizing tab under Recommendation Details.
-
Locate a compute instance in the table for which memory utilization is being collected and expand the row.
-
View the Current and Proposed values on the Memory Utilization card in the details.
-
Navigate to the Cost vs Utilization page and click the EC2 Cost tab.
-
If the memory metric is not displayed, select a Settings file and click the Edit icon.
-
Under Options, change the Metric Statistic to Memory Utilization %.
The Summary chart and Cost Details table now display Memory Utilization %.
-
Either Save the new settings under a new name or Revert the existing file.
The primary AWS account is the one you initially configure in Virtana Platform, that is accessed from the Cloud Providers page. Account linking is used when the primary AWS account is a consolidated billing account. With consolidated billing, linked sub-accounts roll up to the primary for billing purposes.
You should link an account if it contains resources such as EC2 and RDS instances or if it is a member account of your consolidated billing account.
You can link one or more AWS accounts to any AWS primary account. When linked, the primary account you created becomes the "parent" account to the "children".
Linking accounts simplifies your setup process and grants the linked accounts access to the latest cost and right sizing reports.
The primary AWS account assigned to an AWS Integration is intended to provide cost data, as it is usually associated with a consolidated billing account. Linked accounts are intended to provide performance data for any AWS accounts associated with the primary consolidated billing account. If you configured a consolidated billing account as your primary AWS account in Virtana Platform, you can add linked accounts to supply performance data from each of the associated AWS accounts. If you do not have consolidated billing, the first AWS account integration should be sufficient.
Depending on your integration account type, you need to copy authentication values from the AWS IAM role or access key account for each account to be linked. If an IAM role or a user for access keys has not yet been created for the linked account, follow the instructions to create an IAM role or create a read-only user for the account to be linked.
Related Topics
This setup method leverages an AWS CloudFormation Template that creates an IAM role in AWS. The JSON template is accessed from the Cloud Provider Integration setup form in Virtana Platform. Using this template is the simplest and quickest of the three AWS integration setup methods and is recommended over the other setup methods.
You can view a list of permissions granted by the IAM role.
About This Task
When you use the CloudFormation Template to create a new AWS Integration in Virtana Platform, the template creates an AWS stack and populates a read-only IAM role in your AWS account. The IAM role is linked to Virtana Platform using the integration’s Account ID and External ID. Once created, it may take a few minutes for the integration status to be updated.
Prerequisites
-
You must have enabled Cost Explorer and created a Cost and Usage Report in AWS.
-
You must have administrator access to both the Virtana Platform and the AWS consoles.
-
Ensure the AWS Security Token Service (STS) is active for all regions in which you have resources.
If you get the error "Invalid IAM role was rejected" when saving the integration, it indicates STS is inactive in ore or more regions.
Steps
-
In Virtana Platform:
-
Navigate to Settings>Integrations>Cloud Providers.
If this is the first time configuring a cloud account, you will see a page stating Configure Your First Cloud Integration.
-
Click Add Integration and select the appropriate integration type.
-
Optional: Enter a descriptive name for the integration instance to identify its purpose.
If no name is given, Virtana Platform provides a unique default name.
-
-
Under AWS Authentication, select the IAM role authentication type, and then click the link to Open script in AWS.
This opens a new tab in AWS.
Tip
Keep Virtana Platform open to the integration setup.
-
In AWS, do the following:
-
In the Virtana Platform integration setup form, paste the Role ARN value into the IAM Role ARN field and click Save.
Make sure there are no extra spaces after you have pasted the value into the field.
-
In AWS, navigate to AWS Cost Management>Reports and click the name of the report to be used by Virtana Platform as the source of detailed billing data.
-
On the AWS Report Details page, make note of the S3 bucket name and the report path prefix.
-
In Virtana Platform, enter the S3 bucket name and report path prefix in the Enable Detailed Billing Analysis field.
This completes the CloudFormation setup of the primary account. If you intend to configure child linked accounts, keep the Virtana Platform integration configuration form open. If you are not configuring linked accounts, you can close the form.
Tip
After creating your IAM role, wait 2-5 minutes for AWS to finalize its creation before proceeding to the next steps. This ensures the new role has the correct S3 access permissions when added to Virtana Platform.
Next Steps
If you want to add linked accounts, see Create Linked Accounts.
When the CloudFormation Template creates the stack in AWS, a role with the ReadOnlyAccess policy is attached. The following table identifies the access granted by the read-only policy. Most of the services listed are not accessed by Virtana Platform, but to keep setup simpler, Virtana Platform is provided universal read-only access.
Service |
Access Level |
Resource |
---|---|---|
Config |
Full: List Limited: Read |
All resources |
Inspector |
Full: List, Read |
All resources |
Alexa for Business |
Full: List Limited: Read |
All resources |
Amplify |
Limited: List, Read |
All resources |
API Gateway |
Full: Read |
All resources |
App Mesh |
Full: List Limited: Read |
All resources |
Application Auto Scaling |
Full: Read |
All resources |
Application Discovery |
Full: List, Read |
All resources |
AppStream 2.0 |
Full: Read Limited: Write |
All resources |
AppSync |
Full: List, Read |
All resources |
Athena |
Full: List, Read |
All resources |
Auto Scaling |
Full: Read |
All resources |
Backup |
Full: List Limited: Read |
All resources |
Batch |
Full: List, Read |
All resources |
Certificate Manager |
Full: List Limited: Read |
All resources |
Certificate Manager Private... |
Full: List, Read |
All resources |
Cloud Directory |
Full: List, Read |
All resources |
Cloud Map |
Full: List Limited: Read |
All resources |
Cloud9 |
Limited: Read |
All resources |
CloudFormation |
Full: List, Read |
All resources |
CloudFront |
Full: List, Read |
All resources |
CloudHSM |
Full: List, Read |
All resources |
CloudSearch |
Full: List Limited: Read |
All resources |
CloudTrail |
Full: List, Read |
All resources |
CloudWatch |
Full: List, Read |
All resources |
CloudWatch Logs |
Limited: List, Read |
All resources |
CodeBuild |
Full: List Limited: Read |
All resources |
CodeCommit |
Full: List Limited: Read |
All resources |
CodeDeploy |
Full: List, Read |
All resources |
CodePipeline |
Full: List, Read |
All resources |
CodeStar |
Full: List, Read |
All resources |
Cognito Identity |
Full: List, Read |
All resources |
Cognito Sync |
Full: List, Read |
All resources |
Cognito User Pools |
Full: List, Read |
All resources |
Connect |
Full: List Limited: Read |
All resources |
Data Exchange |
Full: List, Read Limited: Write |
All resources |
Data Lifecycle Manager |
Full: List Limited: Read |
All resources |
Data Pipeline |
Full: List, Read |
All resources |
DataSync |
Full: List, Read |
All resources |
Device Farm |
Full: List, Read |
All resources |
Direct Connect |
Full: List, Read |
All resources |
Directory Service |
Full: List, Read |
All resources |
DMS |
Full: List, Read |
All resources |
DynamoDB |
Full: List Limited: Read |
All resources |
DynamoDBAccelerator |
Full: List Limited: Read |
All resources |
EC2 |
Full: Read Limited: List |
All resources |
EC2 Auto Scaling |
Full: List, Read |
All resources |
EC2 Messages |
Full: Read |
All resources |
EFS |
Full: List Limited: Read |
All resources |
EKS |
Full: List, Read |
All resources |
Elastic Beanstalk |
Full: List, Read |
All resources |
Elastic Container Registry |
Full: List, Read |
All resources |
Elastic Container Service |
Full: List, Read |
All resources |
Elastic Transcoder |
Full: List, Read |
All resources |
ElastiCache |
Full: List, Read |
All resources |
Elasticsearch Service |
Full: List, Read |
All resources |
ELB |
Full: List, Read |
All resources |
ELB v2 |
Full: Read |
All resources |
EMR |
Full: List Limited: Read |
All resources |
EventBridge |
Full: List, Read |
All resources |
Firehose |
Full: List |
All resources |
FSx |
Full: Read |
All resources |
GameLift |
Full: List Limited: Read |
All resources |
Glacier |
Full: List, Read |
All resources |
GlobalAccelerator |
Full: List, Read |
All resources |
Glue |
Limited: Read |
All resources |
GuardDuty |
Full: List Limited: Read |
All resources |
Health |
Full: Read Limited: Permissions management |
All resources |
IAM |
Full: List, Read |
All resources |
Import/Export |
Full: List, Read |
All resources |
loT |
Full: List Limited: Read |
All resources |
loT Analytics |
Full: List Limited: Read |
All resources |
loT Greengrass |
Full: List, Read |
All resources |
Kinesis |
Full: List Limited: Read |
All resources |
Kinesis Analytics |
Full: List, Read |
All resources |
Kinesis Analytics V2 |
Full: List, Read |
All resources |
Kinesis Video Streams |
Full: List, Read |
All resources |
KMS |
Full: List, Read |
All resources |
Lambda |
Full: List, Read |
All resources |
Lex |
Full: List, Read |
All resources |
Lightsail |
Full: Read Limited: List |
All resources |
Machine Learning |
Full: List, Read |
All resources |
MediaConvert |
Full: List, Read |
All resources |
MediaPackage |
Full: Read |
All resources |
Migration Hub |
Full: List, Read |
All resources |
Mobile Analytics |
Full: Read |
All resources |
Mobile Hub |
Full: List, Read Limited: Write |
All resources |
MQ |
Full: List, Read |
All resources |
MSK |
Full: List, Read |
All resources |
OpsWorks |
Full: Read Limited: List |
All resources |
OpsworksCM |
Full: List |
All resources |
Organizations |
Full: List, Read |
All resources |
Performance Insights |
Full access |
All resources |
Personalize |
Full: List, Read Limited: Write |
All resources |
Pinpoint |
Limited: List, Read |
All resources |
Pinpoint Email |
Full: List, Read |
All resources |
Polly |
Full: List, Read |
All resources |
RDS |
Full: List, Read |
All resources |
Redshift |
Limited: List, Read |
All resources |
Rekognition |
Full: List Limited: Read |
All resources |
Resource Access Manager |
Full: List, Read |
All resources |
Resource Group Tagging |
Limited: Read |
All resources |
Resource Groups |
Full: List, Read |
All resources |
RoboMaker |
Full: List, Read |
All resources |
Route 53 |
Full: List, Read |
All resources |
Route 53 Resolver |
Full: List, Read |
All resources |
Route53 Domains |
Full: List, Read |
All resources |
S3 |
Full: List Limited: Read |
All resources |
SageMaker |
Full: List Limited: Read |
All resources |
Secrets Manager |
Full: List Limited: Read |
All resources |
SecurityHub |
Full: List, Read |
All resources |
Serverless Application Rep... |
Full: List, Read |
All resources |
Service Catalog |
Full: List Limited: Read |
All resources |
Service Quotas |
Full: Read |
All resources |
SES |
Full: List Limited: Read |
All resources |
Shield |
Full: List, Read |
All resources |
SimpleDB |
Full: List Limited: Read |
All resources |
Snowball |
Full: List, Read |
All resources |
SNS |
Full: List, Read |
All resources |
SQS |
Full: List, Read |
All resources |
Step Functions |
Full: List, Read |
All resources |
Storage Gateway |
Full: List, Read |
All resources |
STS |
Full: Read |
All resources |
SWF |
Full: List, Read |
All resources |
Systems Manager |
Full: List Limited: Read |
All resources |
Transcribe |
Full: List, Read |
All resources |
Transfer |
Full: List, Read |
All resources |
Trusted Advisor |
Full: Read |
All resources |
WAF |
Full: List, Read |
All resources |
WAF Regional |
Full: List, Read |
All resources |
WorkDocs |
Full: List Limited: Read |
All resources |
WorkLink |
Full: List, Read |
All resources |
WorkMail |
Full: List, Read |
All resources |
Workspaces |
Full: Read Limited: List |
All resources |
X-Ray |
Full: Read Limited: Permissions management |
All resources |
To set up an Identity and Access Management (IAM) role, consider using the CloudFormation Template that greatly simplifies creation of the role. However, if you prefer to create the role manually, you can do so.
About This Task
To set up an AWS integration in Virtana Platform using the IAM Role, you must complete tasks in both the AWS Console and in Virtana Platform.
After creating your IAM role, wait 2-5 minutes for AWS to finalize its creation before proceeding to the next steps. This ensures the new role has the correct S3 access permissions when added to Virtana Platform.
You can view a list of permissions granted by the IAM role.
Prerequisites
-
You must have administrator access to both the Virtana Platform and the AWS consoles.
-
You must have enabled Cost Explorer and created a Cost and Usage Report in AWS.
Tip
If you already have an existing IAM role for Virtana Platform but it does not include in-line policies for Cost Explorer or Cost and Usage Reports, you only need to add the policies to the IAM role.
Inline policies provide a one-to-one relationship between a specific policy and a specific user, role, or group. In this case, the inline policy will be embedded in the IAM Role, when that role is created.
-
In your AWS Console, search for IAM, and select the IAM service.
-
In the navigation pane, select Access Management > Policies and click Create Policy.
-
Select the JSON tab and replace the default content with the following code:
{ "Version": "2012-10-17", "Statement": [ { "Action": "ce:Get*", "Resource": "*", "Effect": "Allow" } ] }
-
Click Next: Tags and add any needed tags.
Adding tags is optional.
-
Click Next: Review and provide a descriptive Name for the policy.
-
Make a note of the policy name, review the permissions summary, and click Create Policy.
You will need the policy name to attach this customer managed policy to your IAM role.
Inline policies provide a one-to-one relationship between a specific policy and a specific user, role, or group. In this case, the inline policy will be embedded in the IAM Role, when that role is created.
-
In your AWS Console, search for IAM, and select the IAM service.
The Identity and Access Management (IAM) dashboard displays.
-
In the navigation pane, select Access Management > Policies and click Create Policy.
-
Select the JSON tab and replace the default content with the following code:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "cur:DescribeReportDefinitions", "Resource": "*" } ] }
-
Click Next: Tags and add any needed tags.
Adding tags is optional.
-
Click Next: Review and provide a descriptive Name for the policy.
Example:
ReadCostAndUsageReportDefinitions
-
Make a note of the policy name, review the permissions summary, and click Create Policy.
You will need the policy name to attach this customer managed policy to your IAM role.
AWS Identity and Access Management (IAM) roles provide the ability to grant permissions to trusted entities. IAM roles issue temporary security credentials that are only valid for a role session, providing greater security. The IAM read-only role will be used to allow Virtana Platform modules to access AWS APIs.
About This Task
To create an IAM role, you need to perform the following in AWS:
-
Create the IAM role.
-
Select and configure one of the following options:
-
Standard permissions
The simplest setup. Provides read-only access to everything.
-
Minimal cost permissions
The most secure setup. Provides Virtana Platform access only to services for which Virtana Platform is collecting data. This would include cost reports and performance metrics for the Cloud Cost Management module.
Must be created before being assigned to an IAM Role.
-
Management account permissions
Provides read-only access only for Cost and Usage Reports for a single S3 bucket for a particular customer.
-
Prerequisites
-
You need the Account ID and External ID values for the AWS Integration, located on the integration setup form in Virtana Platform (Settings > Integrations > Cloud Providers > Add Integration).
-
You need the name of the S3 bucket associated with the integration you are creating.
This is found under AWS Services in the Cost & Usage Reports section.
Steps
-
In your AWS Console, search for IAM, and select the IAM service.
The Identity and Access Management (IAM) dashboard displays.
-
In the navigation pane, select Access Management > Roles and click Create Role.
-
For "type of trusted entity", select AWS Account and then select Another AWS Account.
-
Provide the Account ID from your Virtana Platform AWS Integration.
This identifies the Virtana Platform account as the user of this role.
-
Check Require external ID and provide the External ID from your Virtana Platform AWS Integration.
Leave Require MFA unchecked.
-
Click Next: Permissions.
-
Define and assign to the read-only role ONE of the following three role permissions available in AWS.
After you have finished the configuration tasks in the AWS Console, you must complete the integration setup form in Virtana Platform.
About This Task
-
Cost Explorer and Detailed Billing Analysis are both required for billing analysis and are therefore preselected in the setup form. Additionally, Virtana Platform supports multiple AWS data sources with Cost Explorer API enabled.
-
If your AWS account implements consolidated billing, you should add linked accounts during integration setup. Linked accounts are associated with your integration's primary account. They are intended to provide performance data for each account associated with a consolidated billing account. The primary account assigned to an AWS Integration has the ability to send cost data to Cloud Cost Management.
-
This integration’s package will be automatically enabled and provisioned to your account as soon as Virtana Platform receives data from the integration.
Prerequisites
-
During the AWS configuration, you made a note of some AWS information. You must now enter the values in the appropriate fields in Virtana Platform, depending on the configuration method you chose. You will need to enter one or more of the following:
-
S3 bucket name and Report path prefix
-
AWS Access Key ID and Secret Access Key
-
AWS IAM Role ARN
If you don't have the account information needed to complete the integration, see Where to Gather AWS Account Information.
-
Steps
-
In Virtana Platform:
-
Navigate to Settings>Integrations>Cloud Providers.
If this is the first time configuring a cloud account, you will see a page stating Configure Your First Cloud Integration.
-
Click Add Integration and select the appropriate integration type.
-
Optional: Enter a descriptive name for the integration instance to identify its purpose.
If no name is given, Virtana Platform provides a unique default name.
-
-
Under Cost Analysis and AWS Authentication, enter the appropriate values that you collected from AWS:
-
For Enable Detailed Billing Analysis, enter the S3 Bucket Name and Report Path Prefix.
-
If using a GovCloud account, enter the AWS Access Key ID and Secret Access Key.
-
If using the IAM Role, enter the AWS IAM Role ARN value.
-
If using Access Key, enter the AWS Access Key ID and Secret Access Key.
-
-
If you are adding Linked Accounts, enter the appropriate values that you collected from AWS.
-
For the IAM Role, enter the Name of the account and the AWS IAM Role ARN value.
-
For Access Key, enter the Name of the account, the AWS Access Key ID and the Secret Access Key.
-
-
Click Save and close the form.
The new integration now displays in the Cloud Provider Integrations list.
Tip
If you get the error "Invalid IAM role was rejected" when saving the integration, it indicates the AWS Security Token Service (STS) is inactive in ore or more regions.
Steps to check the status of each region, as well as to make a region active, can be found in AWS documentation under the section titled "Activating and deactivating AWS STS in an AWS region".
Related Topics
To set up an AWS Integration using access keys, you need to perform actions in both AWS and in Virtana Platform. In AWS, you must create a read-only user, set permissions for that user, and copy the user security credentials. You then create an AWS Integration in Virtana Platform and enter the credentials you copied from AWS.
You can create a read-only user with standard permissions or with minimal permissions in AWS.
Steps
Important
Leave GovCloud disabled in Virtana Platform unless you have signed up for and are using a GovCloud-restricted AWS account.
AWS GovCloud (US) is a segment of Amazon Web Services cloud offerings that restrict physical and logical administrative access to U.S. citizens only. The region meets the requirements for U.S. International Traffic in Arms Regulations (ITAR), and allows users to move Controlled Unclassified Information (CUI) into the cloud.
See the official AWS GovCloud Guide for more information.
Important
If you enable GovCloud, you must use the access key authentication method when configuring the integration.
Inline policies provide a one-to-one relationship between a specific policy and a specific user, role, or group. In this case, the inline policy will be embedded in the IAM Role, when that role is created.
-
In your AWS Console, search for IAM, and select the IAM service.
-
In the navigation pane, select Access Management > Policies and click Create Policy.
-
Select the JSON tab and replace the default content with the following code:
{ "Version": "2012-10-17", "Statement": [ { "Action": "ce:Get*", "Resource": "*", "Effect": "Allow" } ] }
-
Click Next: Tags and add any needed tags.
Adding tags is optional.
-
Click Next: Review and provide a descriptive Name for the policy.
-
Make a note of the policy name, review the permissions summary, and click Create Policy.
You will need the policy name to attach this customer managed policy to your IAM role.
Inline policies provide a one-to-one relationship between a specific policy and a specific user, role, or group. In this case, the inline policy will be embedded in the IAM Role, when that role is created.
-
In your AWS Console, search for IAM, and select the IAM service.
The Identity and Access Management (IAM) dashboard displays.
-
In the navigation pane, select Access Management > Policies and click Create Policy.
-
Select the JSON tab and replace the default content with the following code:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "cur:DescribeReportDefinitions", "Resource": "*" } ] }
-
Click Next: Tags and add any needed tags.
Adding tags is optional.
-
Click Next: Review and provide a descriptive Name for the policy.
Example:
ReadCostAndUsageReportDefinitions
-
Make a note of the policy name, review the permissions summary, and click Create Policy.
You will need the policy name to attach this customer managed policy to your IAM role.
You must create a read-only user. You can create a user with standard permissions. with minimal permissions, or with management account permissions.
Complete ONE of the following three tasks.
Standard permissions grant blanket read-only access to collect CloudWatch performance metrics and billing files from S3.
-
In your AWS Console, search for IAM, and select the IAM service.
-
In the navigation pane under Access management, click Users.
A list of users displays.
-
Click Add Users and enter a User Name.
Example: VPCMM-user
-
For Select AWS access type, select Access key - Programmatic access.
-
Click Next: Permissions, and then click the Attach existing policies directly tab.
-
Search “readonly,” then select the check box for ReadOnlyAccess.
Be sure to select the checkbox. Selecting the policy name displays details about the policy.
If the policy does not display, you might need to reset the Filter pollicies settings.
-
Attach to the user the 2 in-line policies you previously created:
-
Click Filter policies and select Customer managed.
-
Clear any text in the search field.
-
Find and click the checkbox to select the Cost Explorer in-line policy you created.
Example:
CostExplorerAPIReadOnly
-
Find and click the checkbox to select the Cost and Usage Reports in-line policy you created.
Example:
ReadCostAndUsageReportDefinitions
-
-
Click Next: Tags and add tags if you choose to.
Adding tags is optional.
-
Click Next: Review, review the details, and then click Create User.
-
Immediately download or copy the User Security Credentials.
You need these authentication parameters to complete the AWS integration in Virtana Platform.
Important
You will not be able to access the Secret Access Key again in AWS, so it is recommended that you download and securely save the credentials now.
-
Click Close.
-
In AWS, navigate to AWS Cost Management>Reports and click the name of the report to be used by Virtana Platform as the source of detailed billing data.
-
On the AWS Report Details page, make note of the S3 bucket name and the report path prefix.
This information must be entered in the CCM integration setup form to complete the integration setup.
Next Steps
This setup method grants read-only access to collect CloudWatch performance metrics and billing files. It is limited to only the AWS services for which Virtana Platform Cloud Cost Management (CCM) provides cost reports.
If you want to use a limited read-only access policy, you need to create a custom policy first.
-
In your AWS Console, search for IAM, and select the IAM service.
-
In the navigation pane under Access management, click Policies and click Create Policy.
-
Select the JSON tab and replace the default content with the following code:
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "cloudwatch:Get*", "cloudwatch:List*", "cloudwatch:Describe*", "ec2:Get*", "ec2:Describe*", "elasticloadbalancing:Describe*", "iam:Get*", "rds:Describe*", "rds:List*", "s3:Get*", "s3:List*", "s3:Describe*", "tag:Get*", "tag:Describe*" ], "Effect": "Allow", "Resource": "*" } ] }
-
Click Next: Tags and add tags if you choose to.
Adding tags is optional.
-
Click Next: Review, add a policy Name and Description and verify the Summary.
-
Click Create Policy.
The policy will now be available under Customer Managed Policies.
-
Return to the IAM dashboard in AWS and navigate to the Users section.
-
Click Add Users and enter a User Name.
Example:
VPOptimizeUser
-
Under Select AWS access type, select Access Key - Programmatic access and then click Next: Permissions.
-
Click the tab Attach existing policies directly.
-
Attach to the user the read-only policy and the 2 in-line policies you created:
-
Select Create user without a permissions boundary.
-
Click Next: Tags and add tags if you choose to.
Adding tags is optional.
-
Click Next: Review, verify the details, and then click Create User.
-
Immediately download or copy the Access key ID and Secret access key.
You need these authentication parameters to complete the AWS integration in Virtana Platform.
Important
You will not be able to access the Secret Access Key again in AWS, so it is recommended that you download and securely save the credentials now.
-
Click Close.
-
In AWS, navigate to AWS Cost Management>Reports and click the name of the report to be used by Virtana Platform as the source of detailed billing data.
-
On the AWS Report Details page, make note of the S3 bucket name and the report path prefix.
This information must be entered in the CCM integration setup form to complete the integration setup.
Next Steps
This setup method provides a shared management account with limited access. It grants read-only access to collect billing files from a single s3 bucket that can be located in a management account.
The management account permissions are useful if you store billing files in a shared management account and need to grant Cloud Cost Management (CCM) restricted access to one specific S3 bucket.
Note
CCM only reads the costs for accounts that CCM monitors. Data for unrelated accounts is discarded.
About This Task
As part of this task you will create a policy and a user. You must copy the Cost & Usage Report S3 bucket name, report path prefix, Access Key ID, and Secret Access Key to enter in Virtana Platform.
Prerequisites
You need the name of the AWS bucket associated with your CUR files.
Steps
-
In your AWS Console, search for IAM, and select the IAM service.
-
In the navigation pane, select Access Management > Policies and click Create Policy.
-
Switch to the JSON tab, replace the existing content with the following code, and replace 2 instances of
your-bucket-name
in the code with the name of the bucket associated with your CUR files.Example:
Eng_bucket
There are two instances of your-bucket-name that need replacing.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::your-bucket-name" ] }, { "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::your-bucket-name/*" ] } ] }
-
Click Next: Review and provide a descriptive Name for the policy.
Example:
EngBucketReadOnly
-
Make a note of the policy name, review the permissions summary, and click Create Policy.
You will need the policy name to attach this customer managed policy to your IAM role.
-
Return to the IAM dashboard in AWS and navigate to the Users section.
-
Click Add Users and enter a User Name.
Example:
VPOptimizeUser
-
Under Select AWS access type, select Access Key - Programmatic access and then click Next: Permissions.
-
Click the tab Attach existing policies directly.
-
Attach to the user the read-only policy and the 2 in-line policies you created:
-
Select Create user without a permissions boundary.
-
Click Next: Tags and add tags if you choose to.
Adding tags is optional.
-
Click Next: Review, verify the details, and then click Create User.
-
Immediately download or copy the Access key ID and Secret access key.
You need these authentication parameters to complete the AWS integration in Virtana Platform.
Important
You will not be able to access the Secret Access Key again in AWS, so it is recommended that you download and securely save the credentials now.
-
Click Close.
-
In AWS, navigate to AWS Cost Management>Reports and click the name of the report to be used by Virtana Platform as the source of detailed billing data.
-
On the AWS Report Details page, make note of the S3 bucket name and the report path prefix.
This information must be entered in the CCM integration setup form to complete the integration setup.
Next Steps
After you have finished the configuration tasks in the AWS Console, you must complete the integration setup form in Virtana Platform.
About This Task
-
Cost Explorer and Detailed Billing Analysis are both required for billing analysis and are therefore preselected in the setup form. Additionally, Virtana Platform supports multiple AWS data sources with Cost Explorer API enabled.
-
If your AWS account implements consolidated billing, you should add linked accounts during integration setup. Linked accounts are associated with your integration's primary account. They are intended to provide performance data for each account associated with a consolidated billing account. The primary account assigned to an AWS Integration has the ability to send cost data to Cloud Cost Management.
-
This integration’s package will be automatically enabled and provisioned to your account as soon as Virtana Platform receives data from the integration.
Prerequisites
-
During the AWS configuration, you made a note of some AWS information. You must now enter the values in the appropriate fields in Virtana Platform, depending on the configuration method you chose. You will need to enter one or more of the following:
-
S3 bucket name and Report path prefix
-
AWS Access Key ID and Secret Access Key
-
AWS IAM Role ARN
If you don't have the account information needed to complete the integration, see Where to Gather AWS Account Information.
-
Steps
-
In Virtana Platform:
-
Navigate to Settings>Integrations>Cloud Providers.
If this is the first time configuring a cloud account, you will see a page stating Configure Your First Cloud Integration.
-
Click Add Integration and select the appropriate integration type.
-
Optional: Enter a descriptive name for the integration instance to identify its purpose.
If no name is given, Virtana Platform provides a unique default name.
-
-
Under Cost Analysis and AWS Authentication, enter the appropriate values that you collected from AWS:
-
For Enable Detailed Billing Analysis, enter the S3 Bucket Name and Report Path Prefix.
-
If using a GovCloud account, enter the AWS Access Key ID and Secret Access Key.
-
If using the IAM Role, enter the AWS IAM Role ARN value.
-
If using Access Key, enter the AWS Access Key ID and Secret Access Key.
-
-
If you are adding Linked Accounts, enter the appropriate values that you collected from AWS.
-
For the IAM Role, enter the Name of the account and the AWS IAM Role ARN value.
-
For Access Key, enter the Name of the account, the AWS Access Key ID and the Secret Access Key.
-
-
Click Save and close the form.
The new integration now displays in the Cloud Provider Integrations list.
Tip
If you get the error "Invalid IAM role was rejected" when saving the integration, it indicates the AWS Security Token Service (STS) is inactive in ore or more regions.
Steps to check the status of each region, as well as to make a region active, can be found in AWS documentation under the section titled "Activating and deactivating AWS STS in an AWS region".
Related Topics
If you didn't copy from AWS, or lost, any of the account information you need to configure AWS in Virtana Platform, the location of those details in AWS is provided below.
-
S3 bucket name and Report path prefix:
Go to AWS Billing Dashboard > Cost & Usage Reports (left pane), click the report name
-
AWS Access Key ID and Secret Access Key:
Go to IAM Dashboard > Access management > Users (left pane), click the user name, then click the Security Credentials tab and scroll to the Access Key ID
Reminder: The Secret Access Key is not stored. If you don't have it, you will have to create a new one.
-
AWS IAM Role ARN:
Go to IAM Dashboard > Access management > Roles (left pane), click the role name
Related Topics
You should link an account if it contains resources such as EC2 and RDS instances or if it is a member account of your consolidated billing account.
About This Task
The primary AWS account assigned to an AWS Integration is intended to provide cost data, as it is usually associated with a consolidated billing account. Linked accounts are intended to provide performance data for any AWS accounts associated with the primary consolidated billing account. If you configured a consolidated billing account as your primary AWS account in Virtana Platform, you can add linked accounts to supply performance data from each of the associated AWS accounts. If you do not have consolidated billing, the first AWS account integration should be sufficient.
Prerequisites
Depending on your integration account type, you need to copy authentication values from the AWS IAM role or access key account for each account to be linked. If an IAM role or a user for access keys has not yet been created for the linked account, follow the instructions to create an IAM role or create a read-only user for the account to be linked.
Steps
-
In CCM, navigate to Settings > Cloud Provider Integrations.
-
On the Cloud Provider Integrations page, either select an existing integration instance in the list or click Add Integration and select an integration type.
-
Under Linked Accounts, click Add AWS Account.
-
Select either an IAM Role or an Access Key account type.
-
Enter the required information.
-
For an IAM Role account, enter the values for the Name and IAM Role ARN fields.
-
For an Access Key account, enter the values for the Name, AWS Access Key ID, and AWS Secret Access Key fields.
-
-
Click Save.
Related Topics
When setting up an AWS integration instance, you have the option of adding AWS tags that can limit the AWS entities identified in Virtana Platform. This could be useful if you have entities you do not want included in reports. For example, you might have more compute instances than what you are licensed for in Cloud Cost Management (CCM), so you do not want all compute instances included in CCM reports.
The integration filters are created using AWS tags, which consist of key/value pairs. The tags are created in AWS and help you to identify and organize your resources in AWS. For more information about creating AWS tags, see the AWS Tagging Best Practices and the AWS documentation.
Filter tags can be applied to entities in the following AWS instance types:
-
EC2
-
RDS
-
S3
The supported filter types include the following:
-
Inclusion Filters
Inclusion filters limit the scope to a set of entities that match a given criteria, based on a
tag key
ortag value
.This filter is best used when you have a few, specific items you want selected.
-
Exclusion Filters
Exclusion filters limit the scope to everything except the selected items that match a given criteria. For example, excluding the tag VPN would return a list of all elements that do not include the VPN tag.
This filter is best used when you want the majority of given entities available, except for those with certain attributes or tags.
The Cost Management Account ID and External ID are accessible from the Settings area of Virtana Platform.
-
Click the Settings sprocket icon.
-
Under Integrations, select Cloud Providers and either click Add Integration to add a new AWS integration or select an existing AWS integration.
-
Scroll down to the AWS Authentication section.
-
Copy the Account ID and External ID from the Authentication Parameters section.
Comments
0 comments
Article is closed for comments.