Performance tuning has moved to github from 6.6 onwards:
https://github.com/RedHatSatellite/satellite6-performance-tuning
Pdf: https://access.redhat.com/solutions/4224211
Category Archives: Satelitte
Performance Monitoring of Red Hat Satellite 6 using satperf
Continuous time-series metric collection of Satellite & all Capsules are essential while satellite running at scale.
This post is helps to configure & monitor metrics using satellite-performance
1) Tools:
- Collectd – Daemon to collect System Performance Statistics
- Collects CPU, Memory, Disk, Network, Per Process stats (Regex), Postgresql, mongodb, turbostat, qpid, foreman, DynFlow, Passenger, Puppet, Tomcat, collectd..etc
- Graphite/Carbon
- Carbon receives metrics, and flushes them to whisper database files
- Graphite is webapp frontend to carbon
- Grafana – Visualize metrics from multiple backends.
- Dashboards saved in json and customized by Ansible during deployment
2) Architecture
3) How do i configure performance?
Archit has come up with a nice blog for configuration
Description of metrics collected in satperf:
http://arcolife.github.io/blog/2016/10/05/monitoring-in-satperf-metrics-collection
Monitoring Setup: http://arcolife.github.io/blog/2016/08/22/setting-up-collectd-plugins-for-red-hat-satellite-with-graphite-and-grafana/
4) Example Graphs
4.1 ) Passenger Mem
4.2) Postgresql DB (candlepin & foreman)
4.3) Candlepin DB
4.4) Puppet Registrations
4.2) Dynflow Mem
Thanks to Archit, Jhutar for providing inputs & help!
RedHat Satellite 6.2 Considerations for Large scale deployments
Red Hat Satellite is a complete system management product that allows system administrators to manage the full life cycle of Red Hat deployments across physical, virtual, and private clouds. Red Hat Satellite delivers system provisioning, configuration management, software management, and subscription management- all while maintaining high scalability and security. Satellite 6.2 is third major release of the next generation Satellite with a raft of improvements that continue to narrow the gaps in functionality found in Satellite 5 in many critical areas of the product. This Blog provides basic guidelines and considerations for tuning Red Hat Satellite 6.2 & capsule for Large scale deployments
1) Increase open-files-limit for Apache with systemd on satellite & Capsule server
# cat /etc/systemd/system/httpd.service.d/limits.conf
[Service]
LimitNOFILE=1000000
# systemctl daemon-reload
# katello-service restart
2) Increase open-files-limit for Qpid with systemd on satellite & Capsule server
# cat /etc/systemd/system/qpidd.service.d/limits.conf
[Service]
LimitNOFILE=1000000
# systemctl daemon-reload
# katello-service restart
3) Increase postgresql shared_buffer
While registering content hosts at scale to Satellite server, shared_buffers needs to be set appropriately in postgresql.conf. Recommended: 256 MB
4) Increase postgresql max_connections
When registering content hosts at scale, it is recommended to increase max_connections setting (set to 100 by default) as per your needs and HW profile. For example, you might need to set the value to 200 when you are registering 200 content hosts in parallel.
5) Storage planning for qpid
When you use katello-agent extensively, plan storage capacity for /var/lib/qpidd in advance. Currently, in Satellite 6.2 /var/lib/qpidd requires 2MB disk space per a content hos.
6) Increase open-files-limit for Qpid Dispatch Router with systemd on satellite & Capsule server
# cat /etc/systemd/system/qdrouterd.service.d/limits.conf
[Service]
LimitNOFILE=1000000
# systemctl daemon-reload
# katello-service restart
Special Thanks to Jan Jutar & Archit Sarma for the help to get scale numbers.
Red Hat Satellite 6.2 is available
Red Hat Satellite is a system management solution that enables you to deploy, configure, and maintain your systems across physical, virtual, and cloud environments. Satellite provides provisioning, remote management and monitoring of multiple Red Hat Enterprise Linux deployments with a single, centralized tool. Red Hat Satellite Server synchronizes the content from Red Hat Customer Portal and other sources, and provides functionality including fine-grained life cycle management, user and group role-based access control, integrated subscription management, as well as advanced GUI, CLI, or API access.
Red Hat Satellite 6.2 is available now : https://access.redhat.com/blogs/1169563/posts/2464761
Satellite 6.2 Feature Overview:
Remote Execution
Demo: https://www.redhat.com/en/about/videos/satellite-feature-overview-remote-execution
Details: https://access.redhat.com/articles/2464671
Inter-Satellite Sync
Demo: https://www.redhat.com/en/about/videos/satellite-feature-intersatellite-sync
Details:
Importing Existing Hosts via the Bootstrap Script
demo: https://www.redhat.com/en/about/videos/satellite-feature-overview-bootstrap-script
Details: https://access.redhat.com/articles/2280691
PXE-Less Discovery
Demo: https://www.redhat.com/en/about/videos/satellite-feature-overview-pxeless-discovery
Details: https://access.redhat.com/articles/2464701
Red Hat Satellite 6 Performance Tuning: OS and Web Performance
Redhat Satelitte health check
What is satelitte server?
Satellite 6 is the next generation Linux systems management tool and is the combined product of many open source projects, including Foreman, Katello, Pulp, Candlepin, and Puppet. Since all of these projects bring different featuresand functionality, the performance of your Satellite 6 server is essential.
What is capsule server?
The Red Satellite Capsule Server is a Satellite component that provides federated services to discover, provision, and configure hosts outside of the primary Satellite server. A Satellite Capsule Server provides the following features:
-
Pulp Server/Content Node features, including:
-
Repository synchronization
-
Content delivery
-
Why Healthcheck ?
Once we start scaling up datacenter, so as capsule server and managed nodes, its essential to have satelitte healthcheck tool. It make sure to check all basic services, network ports..etc checked.
Any other tools?
There might be other tools. I find this useful and i customized it for my satelitte scale work. This is not officially supported one.
Usage:
- git clone https://github.com/boogiespook/sat6_healthCheck.git
- cd sat6_healthCheck
- chmod 655 sat6_healthCheck.sh
- Update credentials. mkdir ~/.hammer
chmod 600 ~/.hammer
echo << EOF >> /root/.hammer/cli_config.yml
:foreman:
:host: ‘https://$(hostname -f)’
:username: ‘admin’
:password: ‘password’
EOF” - ./sat6_healthCheck.sh
Example Output:
- #######################################
Satellite 6 Health Check Report
####################################### - ####################
## Checking umask ##
#################### - #######################
## Connection Status ##
####################### - ############################
Checking Disk Partitions
############################” - ###########################
Checking Firewall Rules
###########################” - #######################################
## Checking Satellite Configuration ##
####################################### - #######################
Checking OS Services
#######################” - #######################################
Checking Katello/Satellite Services
######################################
Sync capsule with Satelitte server
On satelitte server: #List capsules get id of capsule to be synced hammer capsule list # get info of capsule [root@sat ~]# hammer capsule info --id 2 [Foreman] Username: admin [Foreman] Password for admin: Id: 2 Name: vmcapsule12 URL: https://vmcapsule12:9090 Features: Puppet, Puppet CA, Discovery, Pulp Node Created at: 2015/07/19 07:07:14 Updated at: 2015/07/19 07:07:14 hammer capsule content lifecycle-environments --id 2 [root@sat ~]# hammer capsule content lifecycle-environments --id 2 [Foreman] Username: admin [Foreman] Password for admin: ---|---------|--------------------- ID | NAME | ORGANIZATION ---|---------|--------------------- 1 | Library | Default Organization ---|---------|--------------------- [root@sat ~]# hammer capsule content add-lifecycle-environment --id 1 --environment-id 1 hammer capsule content remove-lifecycle-environment --id 1 --environment-id 1 hammer capsule content synchronize --id 1 hammer capsule content synchronize --id 1 --environment-id 1
Satelitte server Backup and Restore
1 Stop service
katello-service stop
2 Take Backup
/usr/bin/katello-backup /home/backup #wherever space available
3 Start service
katello-service start
4 Restore
/usr/bin/katello-restore /home/backup
Synchronize the content to the capsule server
Capsule server install: https://goo.gl/Y4ByXk
1) List capsule servers from satellite server
[root@satserver ~]# hammer capsule list
[Foreman] Username: admin
[Foreman] Password for admin:
—|——————————————–|——————————————————–
ID | NAME | URL
—|——————————————–|——————————————————–
1 | satserver.xyz.com | https://satserver.xyz.com:9090
2 | capsule01.xyz.com | https://capsule01.xyz.com:9090
2) List available lifecycle environments to the capsule
[root@satserver ~]# hammer capsule content available-lifecycle-environments –id 2
[Foreman] Username: admin
[Foreman] Password for admin:
—|———|———————
ID | NAME | ORGANIZATION
—|———|———————
1 | Library | Default Organization
2 | devel | Default Organization
—|———|———————
[root@satserver ~]#
id: 2 is my capsule server id.
3) Add life cycle environment to capsule
[root@satserver ~]# hammer capsule content add-lifecycle-environment –environment-id 2 –id 2
4) Added life cycle environment is listed here.
[root@satserver ~]# hammer capsule content lifecycle-environments –id 2
5) Synchronize content.
hammer capsule content synchronize –id 2
It takes a while to sync the content based on network.
Install Redhat satelitte capsule server
1) From satellite server generate certficates for capsule server.
capsule-certs-generate –capsule-fqdn –certs-tar ~/-certs.tar
[root@satserver ~]# capsule-certs-generate –capsule-fqdn capsule01.xyz.com –certs-tar ~/capsule01.xyz.com-certs.tar
Installing Done [100%] […………………………………………………]
Success!
To finish the installation, follow these steps:
1. Ensure that the capsule-installer package is installed on the system.
2. Copy /root/capsule01.xyz.com-certs.tar to the system capsule01.xyz.com
3. Run the following commands on the capsule (possibly with the customized
parameters, see capsule-installer –help and
documentation for more info on setting up additional services):
rpm -Uvh http://staserver.xyz.com/pub/katello-ca-consumer-latest.noarch.rpm
subscription-manager register –org “Default_Organization”
capsule-installer –parent-fqdn “satserver.xyz.com”\
–register-in-foreman “true”\
–foreman-oauth-key “meN959SEtUnFWm2cwDCPhudaUbePW2TJ”\
–foreman-oauth-secret “5tdmdELFfnzA4tDt6oNm9usVKezCmpyS”\
–pulp-oauth-secret “6a87Bp5YyH4YTp5WLxUnkRoXLSerpkbm”\
–certs-tar “/root/capsule01.xyz.com-certs.tar”\
–puppet “true”\
–puppetca “true”\
–pulp “true”
The full log is at /var/log/katello-installer/capsule-certs-generate.log
2) Follow above steps to complete Capsule server installation.
3) Once install done, capsule server will be listed under capsules as shown below.
[root@satserver ~]# hammer capsule list
[Foreman] Username: admin
[Foreman] Password for admin:
—|——————————————–|——————————————————–
ID | NAME | URL
—|——————————————–|——————————————————–
1 | satserver.xyz.com | https://satserver.xyz.com:9090
2 | capsule01.xyz.com | https://capsule01.xyz.com:9090
As shown above, every satellite server will have one default capsule server with id: 1
Pradeep K Surisetty 