|
Server : Apache System : Linux cvar2.toservers.com 3.10.0-962.3.2.lve1.5.73.el7.x86_64 #1 SMP Wed Aug 24 21:31:23 UTC 2022 x86_64 User : njnconst ( 1116) PHP Version : 8.4.18 Disable Function : NONE Directory : /opt/alt/python37/lib/python3.7/site-packages/clsummary/__pycache__/ |
Upload File : |
B
�c1� � @ s� d dl mZ d dl mZ d dlZd dlZd dlmZ d dlZd dlZd dl Z d dl
Z
d dl Z d dl Z d dl
Z
d dlZd dlZd dlZd dlZd dlZd dlmZmZ d dlmZmZ d dlmZ d dlmZ d d l m Z d d
l m Z d d l m!Z! d d l"m#Z# d dl$Z$d dl%Z&d dl'Z'd d
l(m)Z) d dl*m+Z+ d dl,m-Z- d dl.m/Z/m0Z0m1Z1m2Z2m3Z3 d dl4m5Z5 d dl6m7Z7m8Z8m9Z9m:Z:m;Z;m<Z<m=Z=m>Z>m?Z? d dl@mAZAmBZBmCZC d dlDmEZE d dlFmGZGmHZHmIZI d dlJmKZKmLZL d dlMmNZNmOZO d dlPmQZQ d dlRmSZS ddlTmUZU d d lVmWZWmXZXmYZYmZZZm[Z[m\Z\m]Z]m^Z^ d d l_m`Z`maZambZbmcZcmdZdmeZemfZfmgZgmhZhmiZimjZjmkZkmlZlmmZmmnZnmoZompZp d d lqmrZr d d lsmtZtmuZumvZv d d lwmxZx d d!lymzZ{ d d"l|m}Z}m~Z~ d d#lm�Z� d d$l�m�Z� yTd d%l�m�Z�m�Z�m�Z�m�Z�m�Z�m�Z� d d&l�m�Z� d d'l�m�Z� d d!l�mzZ� d d(l�m�Z� W n> e�k
�rj ejZ�ejZ�ejZ�ejZ�ejZ�ejZ�ejZ�dZ�dZ�dZ�Y nX d)Z�e ��d*�Z�d+Z�d,Z�d-Z�d.Z�d/Z�d0Z�d1Z�d2Z�G d3d4� d4e��Z�e�e�d5�d6d7�Z�G d8d9� d9e��Z�d:d;� Z�dS )<� )�absolute_import)�divisionN)�range)� lru_cache�partial)�Counter� defaultdict)�groupby)� iteritems)�
attrgetter)� cpu_count)�getfqdn)�Path)�cpapi)�Feature)�
MySQLGovernor)�is_cl_solo_edition�is_cl_shared_edition�
CLEditions� is_ubuntu�is_secureboot_enabled)�is_panel_feature_supported) �get_rhn_systemid_value� run_command�ExternalProgramFailed�is_testing_enabled_repo�is_litespeed_running�get_cl_version�
get_virt_type�grep�is_nginx_running)�
find_wp_paths�get_wp_cache_plugin� get_wp_paths_with_enabled_module)�
SysCtlConf)�cagefs_statistics_config�db_governor_lib�clconfig_utils)�print_dictionary�replace_params)�LimitsDataStorage�DEFAULTS)� EMPTY_LIMITS)�LvpMap� )� parse_cloudlinux_summary_opts)�Dict�Optional�Callable�Union�List�Any�Tuple�AnyStr)�is_statistic_enabled�is_statistic_rpm_enabled� set_statistic_collection_enabled�$set_statistic_rpm_collection_enabled�$write_statistics_send_status_to_file�
SummaryStatus� is_python_selector_installed�is_ruby_selector_installed� is_nodejs_selector_installed�is_php_selector_installed�dummy_none_function� get_packages_with_lve_extensions�is_sending_process_running�$get_statistics_send_status_from_file� get_client_data_from_jwt_token�get_cl_plus_sender_status� is_active_cloudlinux_license)�get_rpm_packages_info)�get_cpu_metrics�get_memory_metrics� NotSupported)� ALL_MODULES)� PKG_VERSION)� CONFIG_PATH�_read_config_file)� is_email_notification_enabled)� get_hidepid_typing_from_mounts)�get_versions_statistics�get_native_version_safe�get_php_selector_usage�iter_server_applications�get_default_php_version�get_mode_of_php_selector)�CloudlinuxSelectorLib)�get_default_version)� PKG_RELEASEz /var/log/cloudlinux-summary.logzcloudlinux-summary�unknownZ installedZ
not_installedZnot_initializedZ not_selected�enabledZdisabledz-42c
@ sL e Zd ZdZddddgdgddiddiddig d� Zd dd�Zd
d d
�Zd S )�StatisticsDictz>
Special class to store all metrics before sending it
z-42i���g E�) �str�int�float�str_list�int_list�int_dictZ
float_dict�str_dict�
rpm_stat_listTc
C sR | j | }y
|� }W n8 tk
rL } z|r8t�d||� |}W dd}~X Y nX |S )a
Method for catching any exceptions while calling passed function,
logging them and return default result in case if exception is present or
return formatted result in case if exception is absent
:param log_exception: True - exception will been logged, False - exception won't been logged
This flag must be used if param `func` is function, which processes intermediate value, which can equal
to -42 (it means that exception was logged in previous moment) and
doesn't call external functions or doesn't process another values. In other cases
log_exception should be equal True, because in calling of external function or processing of another
value can be raised exception, which should be logged.
Example: functions `lambda: intermediate_value[0].property` or
[len(value) for value intermediate_value.values()] should be used with log_exception equal False,
because they only process intermediate value. But function:
def some_function(intermediate_value):
result = external_function()
return len(intermediate_value), result
should be used with log_exception equal True
z%s. Exception: "%s"N)�DEFAULT_RESULTS� Exception�
app_logger� exception)�self�func�type_of_result� log_message�
log_exceptionZdefault_result�result�err� rr �B/opt/alt/python37/lib/python3.7/site-packages/clsummary/summary.py�run_safe_or_log_errors� s
z%StatisticsDict.run_safe_or_log_errorsc C s | j ||||d�}|| |< dS )z�
Method which call `run_safe_or_log_errors` and save result from it
:param log_exception: True - exception will been logged, False - exception won't been logged
See method `run_safe_or_log_errors`
)ro N)rt )rk rl rm Zname_of_metricrn ro rp rr rr rs �
format_metric� s z StatisticsDict.format_metricN)T)T)�__name__�
__module__� __qualname__�__doc__rg rt ru rr rr rr rs r^ � s
&r^ )� site_dict�returnc C s t tdd� | d �� ��S )zB
Checks if wordpress site has at least one enabled module
c S s | d S )Nr] rr )�modulerr rr rs �<lambda>� � z*site_has_enabled_modules.<locals>.<lambda>�modules)�any�map�values)rz rr rr rs �site_has_enabled_modules� s r� c @ s� e Zd ZdZdZe�dddge�e j
�e�e j �e�e j �g�Z
dZdZdZd d
� Zed d � �Zed
d� �Zedd� �Zedd� �Zedd� �Zedd� �Zdd� Zedd� �Zdd � Z d d � Z ed d � �Z ed!d"� �Z ed#d$� �Z d%d&� Z!d'd(� Z"ed)d*� �Z#d+d,� Z$ed-d.�d/d0��Z%d1d2� Z&ed3d4� �Z'ed5d6� �Z(ee)e*e*f e+d7�d8d9��Z,d:d;� Z-d<d=� Z.ee/e0d-d>�d?d@��Z1e)e0e/f e0e+d-dA�dBdC�Z2dDdE� Z3edFdG� �Z4dHdI� Z5dJdK� Z6edLdM� �Z7dNdO� Z8d-d.�dPdQ�Z9dRdS� Z:edTdU� �Z;dVdW� Z<dXdY� Z=dZd[� Z>ed\d]� �Z?d^d_� Z@d`da� ZAdbdc� ZBddde� ZCdfdg� ZDdhdi� ZEdjdk� ZFdldm� ZGdndo� ZHeeIeJdp�dqdr��ZKeLeJeM eJeM e)eIe/f f d.�dsdt�ZNeIeIe)eIeIf e)eIeOeJeIf f du�dvdw�ZPeIeIdx�dydz�ZQeeRd-d{�d|d}� ��ZSd~d� ZTd�d�� ZUed�d�� �ZVed�d�� �ZWed�d�� �ZXed�d�� �ZYed�d�� �ZZed�d�� �Z[d�d�� Z\ed�d�� �Z]ed�d�� �Z^d�d�� Z_ed�d�� �Z`ed�d�� �Zad�d�� Zbed�d�� �Zcd�d�� Zded�d�� �Zed�d�� Zfed�d�� �Zged�d�� �Zhed�d�� �Ziee+d.�d�d���Zjee+d.�d�d���Zkee/d.�d�d���ZleeId.�d�d���Zmed�d�� �Znd�d�� Zod�d�� Zpe0eqe/ d��d�d��Zrd�d�� Zsd�d�� Ztd�d�� Zuee)d.�d�d�Zvd�dĄ Zwd�dƄ ZxeLe/e/e/f d.�d�dȄZyeeLe/eIf d.�d�dʄ�Zzd�d̄ Z{d�d΄ Z|d�dЄ Z}d�d҄ Z~d�dԄ Zed�dք �Z�d�d Z�e�d�dڄ �Z�ed�d܄ �Z�ed�dބ �Z�d�d� Z�d�d� Z�d�d� Z�ed�d�d��Z�ed�d�d��Z�ed�d� �Z�ed�d� �Z�ed�d� �Z�ed�d� �Z�ed�d�� �Z�d�d�� Z�ed�d�� �Z�d-S )��CloudlinuxSummaryz /var/lve/dashboard_certificatez/etc/cl_plus/.disabled�python�ruby�nodejsz-https://stat-api.cloudlinux.com/api/clos-statz-https://stat-api.cloudlinux.com/api/rpm-statsz6https://repo.cloudlinux.com/static/cl-settings-v1.jsonc C s> t � | _d | _t� | _d | _d | _| �� | _d | _ t
� | _ d S )N) �dict�_opts�_security_tokenr^ �
statistics�_lvpmap�
_system_id�_is_process_not_limited�is_process_not_limited�packages_by_lenr$ �sysctl)rk rr rr rs �__init__ s
zCloudlinuxSummary.__init__c C s, t tj�d��} t tj�d��}| o*| S )zm
Return true if process is running outside LVE or it's not running by
utility `nice`
�RUNNING_BY_NICE�RUNNING_IN_LVE)�bool�os�environ�get)Zis_running_by_niceZis_running_in_lverr rr rs r� s z)CloudlinuxSummary._is_process_not_limitedc C s | j dkrt� | _ | j S )z.
Load lvpmap only when needed
N)r� � _get_lvpmap)rk rr rr rs �lvpmap s
zCloudlinuxSummary.lvpmapc C s | j d krtd�| _ | j S )N� system_id)r� r )rk rr rr rs r� s
zCloudlinuxSummary.system_idc C s t � r
dS dS )NZubuntuZ rhel_like)r rr rr rr rs �
_get_platform' s z CloudlinuxSummary._get_platformc C s t � r
tS tS )N)r �ENABLED�DISABLEDrr rr rr rs �_detect_secureboot+ s z$CloudlinuxSummary._detect_securebootc s, t jt j � d�� fdd�td�D ��} | S )N� c 3 s | ]}t �� �V qd S )N)�randomZchoice)�.0�_)�range_for_random_choicerr rs � <genexpr>2 s z=CloudlinuxSummary._generate_security_token.<locals>.<genexpr>�@ )�stringZ
ascii_lettersZdigits�joinr )�security_tokenrr )r� rs �_generate_security_token/ s z*CloudlinuxSummary._generate_security_tokenc
C s� i }| j d kr*d}t�|� | �d|i� d| j }t�|� | j| j d�}d }ytj| j|dd�}W nD tj k
r� } z$t
|�}t�|� | �d|i� W d d }~X Y nX |j s�d� |j
|j�}t�|� | �d|i� t�d� y2|�� d }t|d t��r|d �| �� � W n~ ttfk
�rL d }t�|� | �d|i� Y nT tk
�r� } z*t�d
t
|�|j� | �dd t i� W d d }~X Y n X t�d � |S )
NzSecurity token is emptyrp z!Getting statistics from server %s)r� r� �< )�params�timeoutz*Server answer is: HTTP code {}; Reason: {}z!Received response from the serverz Can't parse api response to jsonz=Invalid json response from server, field %s not found in "%s"z1Invalid response from server. See %s for details.z,SUCCESS: received statistics from the server)r� ri �error�_error_and_exit� SUMMARY_URL�infor� �requestsr� �RequestExceptionr_ �ok�format� status_code�reason�json�
isinstancer� �update� _get_smart_advice_statistics� TypeError�
ValueError�KeyError�text�LOG_FILE)rk Z stat_data�messager� �response�err rr rs �_get_remote_data5 sP
z"CloudlinuxSummary._get_remote_datac C sL | j d k r| j S tj�| j�r*| �� | _ n | �� }| �|�r@|nd | _ | j S )N)r� r� �path�isfile�DASHBOARD_CERTIFICATE�_read_token_from_filer� �_write_token_to_file)rk �tokenrr rr rs r� l s
z CloudlinuxSummary.security_tokenc
C sp y4t | jd��}|�|� W dQ R X t�| jd� dS ttfk
rj } zt�dt |�� dS d}~X Y nX dS )z�
Write security token to file and return success/fail status
:param token: generated security token
:return: T/F status
�wNi� Tz,Error while writing secure token to file: %sF)
�openr� �writer� �chmod�IOError�OSErrorri r� r_ )rk r� �fr� rr rr rs r� x s z&CloudlinuxSummary._write_token_to_filec
C sb y&t | j��}|�� �� p d S Q R X W n6 ttfk
r\ } zt�dt|�� d S d }~X Y nX d S )Nz.Error while reading file with secure token: %s) r� r� �read�stripr� r� ri r� r_ )rk r� r� rr rr rs r� � s z'CloudlinuxSummary._read_token_from_filec C s t jt jdddd�S )z�
Detect old LVE limits integration presence according to
https://docs.cloudlinux.com/index.html?lve_limits_with_packages.html
:return: True/False - present/absent
ZCUSTOM_GETPACKAGE_SCRIPT�=F)� file_name�
param_nameZ separatorZ default_val)�detectZget_boolean_paramZCL_CONFIG_FILErr rr rr rs �_detect_old_lve_integration� s
z-CloudlinuxSummary._detect_old_lve_integrationc C s t j�d�S )zO
Detects presence/absence of lsapi
:return: True/False
z/usr/bin/switch_mod_lsapi)r� r� �existsrr rr rr rs �_is_lsapi_present� s z#CloudlinuxSummary._is_lsapi_presentc C s� | dkrt � sPtS n<| dkr*t� r&tS tS | dkr>t� sPtS n| dkrPt� sPtS t| �}|dkrdtS | dkr�y|�� d r~tS tS t k
r� tS X n| dkr�|�
� r�tS tS t d� | ���dS ) zP
Get selector status for nodejs, python, ruby and php selectors
r� r� r� �phpN)r� r� Zselector_enabledzUnknown interpreter: {})
r> �
NOT_INSTALLEDr? r� r@ rA rY Zget_selector_statusr� r� Zphp_selector_is_disabledr� r� )� interpreter�librr rr rs �_get_status_of_selector� s* z)CloudlinuxSummary._get_status_of_selectorc
C s� yt �|��� }|S t jk
rX } z&t�dt|�� | �dt|�i� W d d }~X Y n6 tt fk
r� } zt�dt|�� d S d }~X Y nX d S )Nz3Request exception while getting remote settings: %srp z'Error while parsing remote settings: %s)
r� r� r� r� ri r� r_ r� r� r� )rk Z settings_url�settingsr� rr rr rs �_get_remote_settings� s "z&CloudlinuxSummary._get_remote_settingsc
C s� | j �d�rdS ttj�d��}|r(dS | �| j�}|dkr@dS y"|d d }|d d ob| �|�S tk
r� } z&t �
d t |�� | � d
t |�i� W dd}~X Y nX dS ) z-
Return cl-statistics status
z--force-collectTZCL_TEST_SYSTEMNFz
cl-statisticsz
rollout-groupr] z4Error occurred while trying to get rollout group: %srp )
r� r� r� r� r� r� � SETTINGS_URL�
_match_serverr� ri r� r_ r� )rk Zis_test_environmentr� Z
rollout_groupr� rr rr rs �_is_statistics_enabled� s z(CloudlinuxSummary._is_statistics_enabledc C s
t | d�S )N� )r` )� hash_serverrr rr rs �
_to_number� s z CloudlinuxSummary._to_numberc C s@ | j d krdS t�| j �� ��� d d� }| �|�d| dkS )NT� � r )r� �hashlibZsha256�encodeZ hexdigestr� )rk Zurl_numr� rr rr rs r� � s
z CloudlinuxSummary._match_serverN)r{ c C s* d} x | r$t � s$| d8 } t�d� qW dS )zC
Wait for running background process of cl-summary
�2 r. g�������?N)rD �time�sleep)Zretriesrr rr rs � _wait_for_background_process� s z.CloudlinuxSummary._wait_for_background_processc C s | j d s| j d r>t| j d � t�� dd�}t|d� d S | j d sR| j d r|t| j d � t�� dd�}t|d� d S | j d r�t� r�d nd
}t�� |dd �}|�d t� i� t|d� td
� | j � d�r�| �
� }| j |t � d� | j � d��rft
� �s | j � d��s | jddid
d� | �� | j � d��rV| j| j| jdd� t�d� n t| jd� d S | �� �s�tjdt�� d�}t|� | jddid
d� | j � d��r2| j � d��s�t � �s�tjd t�� d�}t|� | jdd id
d� | jd k�r2d }t�|� tjd t t�� d�}t|� | �d|i� | j � d ��r�t� �rZt�� d dd �}n<ttj t�� d d!�� t j d"dd#d$� | � � t�� dd�}t|d� td
� | j d �r| j �r�tdd%��r�t�� d dd �}t|d� td
� nttj t�� d d!�� d S )&N�enable�disable�success)� timestamprp Tz
enable-rpmz disable-rpm�statusZ
collectingZready)r� r� rp Zsending_statusr z
get-remote)�datar8 z rpm-packagesz--force-collectrp z\Sending RPM statistics is disabled by admin. Use --force-collect to ignore admin`s settings.)�
error_codez--sendF)�summary�url� save_statuszRPM statistics sentzkStatistics collection is disabled globally. Please, try again later or contact support if it happens again.)rp r� r� zYCollecting statistics is disabled globally. Use --force-collect to ignore global settingsz_Statistics collection is disabled by admin. Run `cloudlinux-summary enable` and then try again.zXSending statistics is disabled by admin. Use --force-collect to ignore admin`s settings.z8Statistics was not sent, because security token is emptyz�We are not able to collect statistics because we are not able to make a security token. Check %s for details or contact support.z--async)rp r� r� z9/usr/sbin/cloudlinux-summary --send --json &> /dev/null &z /bin/bash)�shell�
executable)� acquire_lock)!r� r: r� r( r; rD r� rE �exitr� r� �_print_result_and_exitr8 r9 r� �_get_rpm_packages_summary� _send_statistics_and_save_statusr� �RPM_PACKAGES_URLri r� r� r= �FAILEDr< r� r� r� Z IN_PROGRESS�
subprocess�runr� r� )rk r� r� rp � status_dictr� rr rr rs �&_actions_before_run_process_as_limited� s�
z8CloudlinuxSummary._actions_before_run_process_as_limitedc C sj t jdddd�}t �|��F}| �dt|�g� tjddt|�dg| ttj d t
� t j �d
�d �S Q R X d S )
z�
Run same command in lve and set environ RUNNING_IN_LVE=1
in order to check it in child process.
:return:
� r i @)Zls_cpuZls_cpusZ
ls_memory_phyz--lve-idz/bin/lve_suwrapperz-nz /usr/sbin/cloudlinux-summary�1)r� ZCPU_DATA)�envN)
� lvectllibZmake_liblve_settingsZ
temporary_lve�extendr_ r �callr� r� r� r� �dumpsZ CPUINFO_DATA)�argsr� Zlve_idrr rr rs �_run_self_in_lve~ s
z"CloudlinuxSummary._run_self_in_lvec C s$ t jddddg| ttjdd�d�S )z�
Run same command using utility `nice` and set environ RUNNING_BY_NICE=1
in order to check it in child process.
:return:
z
/usr/bin/nicez-nZ19z /usr/sbin/cloudlinux-summaryr )r� )r )r r r� r� r� )r rr rr rs �_run_self_by_nice� s z#CloudlinuxSummary._run_self_by_nice)�optsr{ c s8 t � fdd�dD ��rdS t � fdd�dD ��r4dS dS )zK
Check that passed command should run as unlimited process
c 3 s | ]}� | V qd S )Nrr )r� �option)r rr rs r� � s zECloudlinuxSummary._should_run_as_unlimited_process.<locals>.<genexpr>)z rpm-packagesr� z
get-remoter� r� z
enable-rpmz disable-rpmTc 3 s | ]}� | V qd S )Nrr )r� r )r rr rs r� � s )z--sendz--asyncz--jsonz--force-collectF)r� )r rr )r rs � _should_run_as_unlimited_process� s
z2CloudlinuxSummary._should_run_as_unlimited_processc
C s | � |�| _| jr�| �� | �| j�r0td� n�ttj�sN| � |�}t|� nly| �
|�}t|� W nT t j k
r� } z4d| }t
|� t�t�}|j|dd� td� W d d }~X Y nX nH| �| j�r�d}t�� |d�}tj|| jd� t|d� td � | �� d S )
Nr z$failed to run task in lve, error: %sT)�exc_info���z|You shouldn't use env var "RUNNING_IN_LVE" or "RUNNING_BY_NICE" for run of any command except collection of main statistics.)r� rp )Zextrar. )� _parse_argsr� r� r r r� r r �LVEr r r
Z
PyLveError�print�logging� getLoggerrv rj r� ri r� r( �run_get_summary)rk �argv�rcr� Z error_msg�logZerr_msgr� rr rr rs r � s2
zCloudlinuxSummary.runc C s� t � � }| �� t � � | }|| jd< | jd rJ| j�t| j|�ddd� | j�d�rv| j| j| j dd� t
� d � n t | jd� d S )
NZcl_summary_execution_timez--lve-idr_ Zcl_summary_max_mem_usedz,Can't get memory usage by cloudlinux-summaryz--sendT)r� r� r� zMain statistics sent)
r� � _get_summaryr� r� ru r �_get_max_memoryr� r r� ri r� r( )rk Z
start_time� running_timerr rr rs r � s"
z!CloudlinuxSummary.run_get_summary)r� �summary_resultr{ c C s2 t j| d�}|dkr&||d< t j|d< t|� dS )z@
Save status of sending statistics to json file
)rp r� r� r� rp N)r= �SUCCESSr r< )r� r" r rr rr rs � _save_status s
z CloudlinuxSummary._save_status)r� r� r� r{ c C sJ t t�� �}||d< | j||d�}||d�}t|d� |rF| �||� dS )zH
Send statistics data to server and save status to file
r� )r� )rp r� TN)r` r� �_send_statisticsr( r$ )rk r� r� r� r� Zs_resultrp rr rr rs r s
z2CloudlinuxSummary._send_statistics_and_save_statusc
C s� |d }|dk rd S dddt | jd �ddd d
�tt�|���g}yt|d d �\}}}W n. tk
r� } zt� d
|� d S d }~X Y nX |dkr�t
� |�}yt dd� |d D ��S t
tfk
r� d S X t�d||� d S )Nr� r. z/usr/sbin/lveinfoz--jsonz--idz--lve-idz--show-columns�mPMemz--periodz{}mT)�return_full_outputz Unable to run lveinfo, error: %sr c S s g | ] }|d �qS )r& rr )r� �xrr rr rs �
<listcomp>/ s z5CloudlinuxSummary._get_max_memory.<locals>.<listcomp>r� z-lveinfo failed with exit code: %i, output: %s)r_ r� r� r` �mathZceilr r ri �warningr� �loads�maxr� r� r� ) rk r! Z time_minutes�cmdr �json_strr� r� Z parsed_datarr rr rs r s*
z!CloudlinuxSummary._get_max_memoryc C s� d}y�d| }t �|� t�d�}xbtd�D ]R}ytj|| dd�}W n6 tjk
rz } z|}t�d| � W dd}~X Y q,X P q,W |�|j d kr�t �d
� nd �
|j |j �}t �|� W n6 tjk
r� } zt |�}t �
|� W dd}~X Y nX |S ) z�
Sends statistics to server
:param data: Statistics data dict
:return: string - message for JSON 'result' key
r� z Sending statictics to server %sz*Unknown exception while sending statistics� r� )r� r� � N�� zSending statictics OKz*Server answer is: HTTP code {}; Reason: {})ri r� r� r� r Zpost�ConnectionErrorr� r� r� r� r� r_ r� )r� r� Z out_messager� Z expected_err�ir� rq rr rr rs r% 6 s0
z"CloudlinuxSummary._send_statisticsc C s& dt � � d�}| �� | j�|� |S )Nr. )�versionr� )r� � _prepare_cl_normal_statisticsr� r� )rk rp rr rr rs r ^ s
z CloudlinuxSummary._get_summaryc C s& dt � � d�}| �� | j�|� |S )Nr. )r5 r� )r� �'_fill_dict_with_rpm_packages_statisticsr� r� )rk rp rr rr rs r� f s
z+CloudlinuxSummary._get_rpm_packages_summaryc C s t �� t jS )z/
Get version of control panel
)r� ZgetCPZ
CP_VERSIONrr rr rr rs �_get_panel_versionn s z$CloudlinuxSummary._get_panel_versionc C s` | j �tjdd�}|dk rR|| j d< | j �dd� ddd� | j �d d� dd
d � n
t| j d
< dS ) zN
Fill dict with statistics by statistics about MySQL governor
r_ z Can't get MySQL governor modeNZmysql_governor_modec S s
t � �� S )N)r Zget_governor_versionrr rr rr rs r} � r~ zCCloudlinuxSummary._fill_mysql_governor_statistics.<locals>.<lambda>Zmysql_governor_versionz Can't get MySQL governor versionc S s t � �� d S )Nr )r Zget_governor_statusrr rr rr rs r} � r~ Zmysql_governor_statusz Can't get MySQL governor status)r� rt r& Zget_gov_mode_operationru r� )rk Zmysql_gov_moderr rr rs � _fill_mysql_governor_statisticsx s"
z1CloudlinuxSummary._fill_mysql_governor_statisticsc C s~ | j �tjddd� | j �dd� ddd� | j �| jdd d
� | j �| jdd d � | j �td
dd� | j �dd� d
dd� dS )zM
Fill dict with statistics by statistics about control panel
r_ �control_panel_namez Can't get control panel namec S s dd� t �� �� D �S )Nc S s g | ]\}}|r|�qS rr rr )r� �nameZ is_supportedrr rr rs r) � s zVCloudlinuxSummary._fill_control_panel_statistics.<locals>.<lambda>.<locals>.<listcomp>)r Zget_supported_cl_features�itemsrr rr rr rs r} � r~ zBCloudlinuxSummary._fill_control_panel_statistics.<locals>.<lambda>rb Zsupported_cl_featuresz8Can't get list of supported cl features by control panelZcontrol_panel_versionz Can't get control panel versionZcontrol_panel_apachez Can't get control panel apacher` Z
nginx_runningz Can't get control panel nginxc S s t t�� �S )N)�lenr �cpusersrr rr rr rs r} � r~ Z users_totalzCan't get amount of usersN)r� ru r� � getCPNamer8 �_get_control_panel_apacher )rk rr rr rs � _fill_control_panel_statistics� s< z0CloudlinuxSummary._fill_control_panel_statisticsc C sP t � r(t�� rd}qLt�� r"d}qLd}n$| jd dkrHt�� rBdnd}nd}|S ) z�
Wrapper to retrieve control panel Apache version:
EA3 or EA4 for cPanel, native otherwise
:return: EA3|EA4|native
Z litespeedZ
openlitespeedZunknown_litespeedr: ZcPanelZEA4ZEA3�native)r r� Zdetect_enterprise_litespeedZdetect_open_litespeedr� Zis_ea4)rk rp rr rr rs r@ � s z+CloudlinuxSummary._get_control_panel_apachec C s8 t �� } | dkr| S t jtt jtdtdti}|�| d�S )z�
Wrapper to convert internal values from cagefs_statistics_config.get_cagefs_status function to values
for statistics
:return:
NZEnabledZDisabledZUnknown) r% Zget_cagefs_statusZCAGEFS_STATUS_NOT_INSTALLEDr� Z CAGEFS_STATUS_NOT_INITIALIZED�NOT_INITIALIZEDr� r� r� )�
cagefs_statusZcagefs_status_maprr rr rs �_cagefs_status_wrapper� s z(CloudlinuxSummary._cagefs_status_wrapperc C s~ | j �| jddd� | j d ttgkr2d| j d< n| j �tjddd� | j �ttjd�ddd � | j �ttjd
�dd d � dS )
zF
Fill dict with statistics by statistics about CageFS
r_ rD zCan't get CageFS statusNZcagefs_user_modezCan't get CageFS user modeTZcagefs_enabled_quantityz/Can't get quantity of users with enabled CageFSFZcagefs_disabled_quantityz0Can't get quantity of users with disabled CageFS) r� ru rE r� rC r% Zget_cagefs_user_moder Z get_quantity)rk rr rr rs �_fill_cagefs_statistics� s,
z)CloudlinuxSummary._fill_cagefs_statisticsc s^ y t dd� | j�� D ��}W n tjk
r2 dS X t| jj�� �� t� fdd�|� � D ��S )zJ
Get amount of end-users which belong to active resellers
c s s | ]\}}|d kr|V qdS )r Nrr )r� r� �lvprr rr rs r� s zLCloudlinuxSummary._get_amount_of_endusers_under_resellers.<locals>.<genexpr>Nc 3 s | ]}|� kV qd S )Nrr )r� Zlvp_id)�enabled_lvp_idrr rs r� s )
r r� �
lve_lvp_pairsr rL �set�name_mapZid_list�sum�elements)rk � lvp_countrr )rH rs �'_get_amount_of_endusers_under_resellers s z9CloudlinuxSummary._get_amount_of_endusers_under_resellersc C s@ y t dd� | j�� D ��}W n tjk
r2 dS X t|�� �S )z/
Get total amount of end-users
c s s | ]\}}|d kr|V qdS )r Nrr )r� r� rG rr rr rs r� s zBCloudlinuxSummary._get_total_amount_of_endusers.<locals>.<genexpr>N)r r� rI r rL rL r� )rk rN rr rr rs � _get_total_amount_of_endusers s
z/CloudlinuxSummary._get_total_amount_of_endusersc C s( y t t�� �S tjk
r" Y nX dS )z)
Get amount of resellers
N)r= r � resellersrL rr rr rr rs �_get_amount_of_resellers s z*CloudlinuxSummary._get_amount_of_resellersc s� � j �tjjddd� � j �tjjddd� � j �tjjddd� � j �� fdd �dd
d � � j �� jdd d
� � j �� jddd� � j �� j ddd� � j �� fdd �ddd� � j d rԈ j �dd � ddd� n d� j d< d� j d< d� j d< dS )zP
Fill dict with statistics by varied statistics about resellers
r` Z reseller_limits_supported_kernelz8Can't detect status of support reseller limits by kernelZ'reseller_limits_supported_control_panelz?Can't detect status of support reseller limits by control panelZreseller_limits_enabledz1Can't detect status of support of reseller limitsc s � � � d S )Nr )�#get_users_and_resellers_with_faultsrr )rk rr rs r} F r~ z>CloudlinuxSummary._fill_resellers_statistics.<locals>.<lambda>Zusers_with_faultsz6Can't get amount of users with faults for the past 24hZresellers_totalz#Can't get total amount of resellersZ(resellers_endusers_under_reseller_limitsz>Can't get amount of end-users which belong to active resellersZresellers_endusers_totalz#Can't get total amount of end-usersc s � � � d S )Nr. )rS rr )rk rr rs r} ^ r~ Zresellers_with_faultsz:Can't get amount of resellers with faults for the past 24hc S s t tt�� ��S )N)r= �listr
Zlvp_listrr rr rr rs r} e r~ Zresellers_activez$Can't get amount of active resellersN)
r� ru r
ZlveZis_lve10Zis_panel_supportedZreseller_limit_supportedrR rO rP )rk rr )rk rs �_fill_resellers_statistics+ sb
z,CloudlinuxSummary._fill_resellers_statisticsc s | j �t| j� jt j� jt j�ddd� | j �t| j� jt j�ddd� | j �� fdd�dd d
� | j �� fd d�dd d
� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �t| j � jt j
�ddd� | j �t| j � jt j �ddd� dS ) zN
Fill dict with statistics by statistics about default limits
r` Zdefault_limit_speedz Can't get default speed limitr_ Z default_limit_cpu_origin_unitsz+Can't get cpu origin units of default limitc s � j t jS )N)�defaultsr+ �ncpurr )�xml_cfg_providerrr rs r} � r~ zCCloudlinuxSummary._fill_default_limits_statistics.<locals>.<lambda>Zdefault_limit_ncpuz Can't get default ncpu limitc s � j t jS )N)rV r+ �iorr )rX rr rs r} � r~ Zdefault_limit_iozCan't get default io limitc s � j t jS )N)rV r+ �nprocrr )rX rr rs r} � r~ Zdefault_limit_nprocz Can't get default nproc limitc s � j t jS )N)rV r+ �eprr )rX rr rs r} � r~ Zdefault_limit_epzCan't get default ep limitc s � j t jS )N)rV r+ �iopsrr )rX rr rs r} � r~ Zdefault_limit_iopsz Can't get default iops limitZdefault_limit_vmem_mbz Can't get default vmem limitZdefault_limit_pmem_mbz Can't get default pmem limitN) r� ru r �_cpu_limit_to_percentsrV r+ �cpurW �_get_cpu_limit_units�_mempages_to_mb�vmem�pmem)rk rX rr )rX rs � _fill_default_limits_statisticso s`
z1CloudlinuxSummary._fill_default_limits_statisticsc sR | j �� fdd�ddd� | j �� fdd�ddd� | j �� fd d�dd
d � d S )
z�
Fill dict with statistics by other statistics about limits:
packages_total, users_total, amount users/packages with custom limits
c s
t � j�S )N)r= �packagesrr )rX rr rs r} � r~ zACloudlinuxSummary._fill_other_limits_statistics.<locals>.<lambda>r` Zpackages_totalz"Can't get total amount of packagesc s t � �� �S )N)r= Z get_packages_with_custom_limitsrr )rX rr rs r} � r~ Zpackages_with_custom_limitsz/Can't get amount of packages with custom limitsc s t � �� �S )N)r= Z get_users_with_custom_limitsrr )rX rr rs r} � r~ Zusers_with_custom_limitsz,Can't get amount of users with custom limitsN)r� ru )rk rX rr )rX rs � _fill_other_limits_statistics� s
z/CloudlinuxSummary._fill_other_limits_statisticsc s� �x�t dd�D �]�}� j�t� j||�dd| ���dkr<P � jj�fdd�dd| d | d
d � � jj�fd d�d
d| d| d
d � � jj� �fdd�d
d| d| d
d � � jj� �fdd�dd| d| d
d � � jj�fdd�d
d| d| d
d � � jj�fdd�d
d| d| d
d � � jj�fd d�d
d | d | d
d � � jj�fd d�d
d | d!| d
d � � jj�fd"d�d
d#| d$| d
d � � jj� �fd%d�d
d&| d'| d
d � � jj� �fd(d�d
d)| d*| d
d � qW dS )+zV
Fill dict with statistics by statistics about top packages on server
r. r1 r_ z!Can't get top %s package by usersNc s
� d j S )Nr. )r; rr )�
top_resultrr rs r} � r~ zACloudlinuxSummary._fill_top_packages_statistics.<locals>.<lambda>ztop_%s_package_namez1Can't get package name of top %s package by usersF)ro c s t � d �S )Nr )r` rr )rf rr rs r} � r~ r` ztop_%s_package_users_numz4Can't get amount of users in top %s package by usersc s$ � � �d jptj�d jp tj�S )Nr. )r] �limitsr, r^ rW rr )rk rf rr rs r} � s ztop_%s_package_limit_speedz0Can't get speed limit of top %s package by usersc s � � �d jptj�S )Nr. )r_ rg r, r^ rr )rk rf rr rs r} � s z%top_%s_package_limit_cpu_origin_unitsz>Can't get cpu origin units of limit of top %s package by usersc s � d j p tjS )Nr. )rg r, rW rr )rf rr rs r} r~ ztop_%s_package_limit_ncpuz/Can't get ncpu limit of top %s package by usersc s � d j p tjS )Nr. )rg r, rY rr )rf rr rs r} r~ ztop_%s_package_limit_ioz-Can't get io limit of top %s package by usersc s � d j p tjS )Nr. )rg r, rZ rr )rf rr rs r} r~ ztop_%s_package_limit_nprocz0Can't get nproc limit of top %s package by usersc s � d j p tjS )Nr. )rg r, r[ rr )rf rr rs r} r~ ztop_%s_package_limit_epz-Can't get ep limit of top %s package by usersc s � d j p tjS )Nr. )rg r, r\ rr )rf rr rs r} r~ ztop_%s_package_limit_iopsz/Can't get iops limit of top %s package by usersc s � � �d jptj�S )Nr. )r` rg r, ra rr )rk rf rr rs r} $ r~ z top_%s_package_limit_vmem_mbz/Can't get vmem limit of top %s package by usersc s � � �d jptj�S )Nr. )r` rg r, rb rr )rk rf rr rs r} , r~ z top_%s_package_limit_pmem_mbz/Can't get pmem limit of top %s package by users)r r� rt r �#_get_top_package_by_number_of_usersru )rk rX r4 rr )rk rf rs � _fill_top_packages_statistics� s�
z/CloudlinuxSummary._fill_top_packages_statisticsc C s( t � }| �|� | �|� | �|� dS )zN
Fill dict with statistiscs by varied statistics about limits
N)r* rc re ri )rk rX rr rr rs �_fill_limits_statistics4 s
z)CloudlinuxSummary._fill_limits_statisticsc sX | j �| jdd�� � dk �rJ| j j� fdd�ddddd � | j j� fd
d�dd d dd � | j j� fd
d�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fd d�d d d dd � | j j� fd d�d!d"d#dd � | j j� fd$d�dd%d&dd � n
t| j d< dS )'zI
Fill dict with statistics by statistics about mod_lsapi
r_ z Can't get raw mod_lsapi infoNc s � d d S )N�criur� rr rr )�raw_lsapi_inforr rs r} J r~ z:CloudlinuxSummary._fill_lsapi_statistics.<locals>.<lambda>Zlsapi_criu_service_statusz Can't get status of criu serviceF)ro c s � d d S )Nrk r5 rr rr )rl rr rs r} Q r~ Zlsapi_criu_service_versionz!Can't get version of criu servicec s � d d S )N� lsapiConfZ
lsapi_criurr rr )rl rr rs r} X r~ Zlsapi_option_criuzCan't get state of criuc s � d d S )Nrm Zlsapi_with_connection_poolrr rr )rl rr rs r} _ r~ Z lsapi_option_connection_poolz,Can't get state of mod_lsapi connection poolc s � d S )NZ
libVersionrr rr )rl rr rs r} f r~ Zlsapi_lib_versionz"Can't get version of mod_lsapi libc s � d S )NZ modStatusrr rr )rl rr rs r} m r~ Zlsapi_mod_statuszCan't get mod_lsapi statusc s � d S )NZ
modVersionrr rr )rl rr rs r} t r~ Zlsapi_mod_versionzCan't get mod_lsapi versionc s � d S )NZ totalDomainrr rr )rl rr rs r} { r~ r` Zlsapi_total_domain_countz5Can't get total amount of domains which use mod_lsapic s � d S )NZ
domainStatrr rr )rl rr rs r} � r~ rd Zlsapi_domain_statz.Can't get statistics of domains with mod_lsapic s � d S )NZ controlPanelrr rr )rl rr rs r} � r~ Zlsapi_apache_environmentz Can't get apache environment)r� rt �get_raw_lsapi_inforu r� )rk rr )rl rs �_fill_lsapi_statistics> s�
z(CloudlinuxSummary._fill_lsapi_statisticsc C s | j �| jddd� d S )Nre Z
wmt_configz!Cant get status of wmt_api_config)r� ru �_get_wmt_api_config)rk rr rr rs � _fill_wmt_settings_statistics� s
z/CloudlinuxSummary._fill_wmt_settings_statisticsc
sp y| � � \�}�W nB tk
rT } z$t�dt|�� g g i �}�W d d }~X Y nX tt��t|� �}y| �|�\�� W n< tk
r� } z t�dt|�� i i �� W d d }~X Y nX |rЈ�dt |�i� | j
� �fdd�ddd� | j
� �fd d�d
d d � | j
� � fd
d�d
dd� | j
� �fdd�ddd� | j
� � fdd�ddd� | j
� �fdd�d
dd� d S )Nz+Error while getting WordPress statistics %sz3Error while getting WordPress plugins statistics %sZ cloudlinux_wpos_object_cachec s t � �S )N)r= rr )� all_wp_pathsrr rs r} � r~ z7CloudlinuxSummary._fill_wp_statistics.<locals>.<lambda>r` Zwordpress_installed_totalz,Can't get total count of installed wordpressc s � S )Nrr rr )�object_cache_pluginsrr rs r} � r~ rd Z wordpress_object_cache_pluginsz&Can't get WP object cache plugins infoc s � S )Nrr rr )�advanced_cache_pluginsrr rs r} � r~ Z wordpress_advanced_cache_pluginsz(Can't get WP advanced cache plugins infoc s t � �� �S )N)rL r� rr )rs rr rs r} � r~ Z$wordpress_object_cache_plugins_totalz,Can't get WP object cache total plugins infoc s t � �� �S )N)rL r� rr )rt rr rs r} � r~ Z&wordpress_advanced_cache_plugins_totalz.Can't get WP advanced cache total plugins infoc s � S )Nrr rr )�wp_php_versionsrr rs r} � r~ Zwordpress_php_versionsz Can't get WP php version info) �_prepare_wp_statisticsrh ri r� r_ rT rJ � _prepare_wp_plugins_statisticsr� r= r� ru )rk Z paths_with_wpos_object_cacher� Zwithout_wpos_object_cacherr )rt rr rs ru rs �_fill_wp_statistics� sX "
z%CloudlinuxSummary._fill_wp_statisticsc C s | � d|�}| � d|�}||fS )zq
Prepares dict with plugin-counter pairs
e.g: {'w3-total-cache': 4, 'redis': 1, 'ls': 2}
z object-cachezadvanced-cache)�_get_plugin_info)rk �wp_pathsrs rt rr rr rs rw � s z0CloudlinuxSummary._prepare_wp_plugins_statistics)� plugin_typerz c sF t td � fdd�|D ���}tt�}x |D ]}|| d7 < q*W |S )Nc s g | ]}t |� ��qS rr )r" )r� r� )r{ rr rs r) � s z6CloudlinuxSummary._get_plugin_info.<locals>.<listcomp>r. )rT �filterr r` )r{ rz rs Zobject_plugin_counter_pairsZpluginrr )r{ rs ry � s
z"CloudlinuxSummary._get_plugin_infoc
C s� g g }}t t�}t�� }x�|D ]�}yt�|�}W n tk
rH w Y nX t t�}x*|D ]"\}} tj� | �rX|| �
|� qXW xb|D ]Z} | � || |�}
|
d } |
d } |� | � |� |
d � | r�| dkr�|| t
| �7 < q�W q W |||fS )z�
Return Wordpress absolute paths, paths with install WPOS object cache module
and counter of php versions that are used by these Wordpresses.
rz � php_version�'wp_paths_with_enabled_wpos_object_cache� undefined)r r` r r> � userdomainsrh rT r� r� r� �append�_get_docroot_wp_infor r= )
rk rr Z$paths_with_enabled_wpos_object_cacheZ php_versionsZ panel_users�userZ domains_info�docroot_domains_map�domain�docroot� docroot_infoZdocroot_wp_pathsZdocroot_php_versionrr rr rs rv � s,
z(CloudlinuxSummary._prepare_wp_statistics)r� r� r� r{ c s� g g dd�}� �fdd�t � �D �}|s,|S � fdd�|D �|d< � fdd�t||�D �|d< x,�� D ] }| �|�}|dkrh||d < P qhW |S )
zM
Return data about docroot's WP paths and used php versions.
r )rz r~ r} c s4 g | ],}t j�� |�� ks,t j�� |��kr|�qS rr )r� r� r� )r� Zwp_path)r� r� rr rs r) s z:CloudlinuxSummary._get_docroot_wp_info.<locals>.<listcomp>c s g | ]}t � ��|��qS rr )r �joinpath)r� r� )r� rr rs r) # s rz c s" g | ]}|d k rt � ��|��qS )N)r r� )r� r� )r� rr rs r) % s r~ r} )r! r# �_get_php_version_for_domain)rk r� r� r� r� rz r� r} rr )r� r� rs r� s
z&CloudlinuxSummary._get_docroot_wp_info)r� r{ c
C s� t �� tjtjtjfkr dS | �� \}}}|�|i �}|�dd�}t �� tjkr\||kr\|S x:|�� D ].\}}||krf|dkrfd|� dd�� �} | S qfW t �� tjkr�d|� dd�� �}
|
S t �� tjkr�|�dd�} |d t
| � � � d
�} | p�dS |S ) z�
Return php version that is used for specified domain.
Determine whether of MultiPHP Manager or PHP Selector is used.
r Zphp_version_idrB zalt-php�.r� zda-phpZ handler_typeN�-) r� r? r Z CPANEL_NAMEZDIRECTADMIN_NAMEZ
PLESK_NAME�_get_system_php_infor� r< �replacer= r� )
rk r� Zsystem_default_php_version�domains_php_info�selector_php_infoZphp_infoZmulti_php_versionr5 �domainsZalt_php_versionZda_php_versionZhandlerr} rr rr rs r� 1 s( z-CloudlinuxSummary._get_php_version_for_domain)�maxsizec C sX yt �� d } W n tk
r( d} Y nX t �� }t� }|dk rJ|d }ni }| ||fS )z�
Return info about:
- system_php_version (implemented only on cPanel)
- domains_php_info (php versions and handlers turned on in MultiPHP)
- selector_php_info (php version used by panel users via PHP Selector)
Zdefault_version_idN�domains_by_php_version)r Zget_system_php_inforh Zget_domains_php_inforU )Zsystem_php_versionr� Zphp_selector_usager� rr rr rs r� X s
z&CloudlinuxSummary._get_system_php_infoc s0 | j �| jdd�� � dk �r"| j �� fdd�ddd� | j �� fd d�dd
d � | j �� fd d�dd
d� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fd d�dd d � | j �� fd d�dd d!� | j �� fd"d�d#d$d%� n
t| j d
< dS )&z/
Fill dict with SSA statistics
r_ zCan't get SSA statisticsNc s � d S )N�configrr rr )� ssa_statsrr rs r} } r~ z8CloudlinuxSummary._fill_ssa_statistics.<locals>.<lambda>re Z
ssa_configzCan't get SSA configc s � d S )Nr5 rr rr )r� rr rs r} � r~ Z ssa_versionzCan't get version of SSAc s � d S )Nr� rr rr )r� rr rs r} � r~ Z
ssa_statuszCan't get state of SSAc s � d S )NZ agent_statusrr rr )r� rr rs r} � r~ Zssa_agent_statusz Can't get state of SSA agentc s � d d S )N� autotracingr� rr rr )r� rr rs r} � r~ Zautotracing_statusz"Can't get state of SSA Autotracingc s � d d S )Nr� Zdisabled_users_quantityrr rr )r� rr rs r} � r~ r` Z#autotracing_disabled_users_quantityz1Can't get state of SSA Autotracing disabled usersc s � d d S )Nr� Z
rules_versionrr rr )r� rr rs r} � r~ Zautotracing_rules_versionz'Can't get SSA Autotracing rules versionc s � d d S )Nr� Zurls_processedrr rr )r� rr rs r} � r~ Zautotracing_urls_processedz.Can't get SSA Autotracing processed URLs countc s � d d S )Nr� Z
urls_selectedrr rr )r� rr rs r} � r~ Zautotracing_urls_selectedz-Can't get SSA Autotracing selected URLs countc s � d d S )Nr� Zrejectsrr rr )r� rr rs r} � r~ rd Zautotracing_rejectsz!Can't get SSA Autotracing rejects)r� rt �
get_ssa_statsru r� )rk rr )r� rs �_fill_ssa_statisticsp sp
z&CloudlinuxSummary._fill_ssa_statisticsc s@ | j �dd� dd�}|dk �r2| j �t| jd�ddd � | j jt| j|t�d
d d d
d� | j jt| j|t�d
ddd
d� | j �tddd� | j �t ddd� | j �t
d�j ddd� | j �t ddd� | j �t
dd�� � dkr�d| j d< d| j d< n<| j j� fd d�d dd d
d� | j j� fd d�d dd d
d� n
t| j d< dS )!zS
Fill dict with statistics by varied statistics about PHP selector
c S s t d�S )Nr� )rS rr rr rr rs r} � r~ zACloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>r` z+Can't get statistics about PHP interpretersNr� r_ Zselector_php_statusz Can't get status of PHP selectorrb Z selector_php_versions_installedz8Can't get list of versions of installed PHP interpretersF)ro Z selector_php_versions_enabledz6Can't get list of versions of enabled PHP interpretersZ selector_php_version_defaultz,Can't get default version of PHP interpreterZselector_php_version_nativez+Can't get native version of PHP interpreterZselector_php_enabled_uiz%Can't get state of UI of PHP selectorZselector_php_modez Can't get mode of PHP selectorz'Can't get summary usage of PHP selectorZ'selector_php_num_domains_by_interpreterZ%selector_php_num_users_by_interpreterc s dd� � d � � D �S )Nc S s i | ]\}}t |�|�qS rr )r= )r� �vr� rr rr rs �
<dictcomp>� s zUCloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>.<locals>.<dictcomp>r� )r< rr )�php_usage_summaryrr rs r} � r~ rd zBCan't get amount of domains which use PHP selector per PHP versionc s dd� � d � � D �S )Nc S s i | ]\}}t |�|�qS rr )r= )r� r� r� rr rr rs r� s zUCloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>.<locals>.<dictcomp>Zusers_by_php_version)r< rr )r� rr rs r} r~ z@Can't get amount of users which use PHP selector per PHP version)r� rt ru r r� �"_get_list_versions_of_interperters� INSTALLEDr� rW rT rY Zphp_selector_is_enabledrX rU r� )rk Zphp_interpretersrr )r� rs � _fill_php_selector_statistics� s|
z/CloudlinuxSummary._fill_php_selector_statisticsc C s | dk s|dk rdS | | S )z�
Get average amount of applications per domain
:param total_apps: total amount of applications
:param amount_of_apps_per_domain: amount of applications per domain
r. Nrr )�
total_appsZamount_of_apps_per_domainrr rr rs � _get_average_apps_per_domain s z.CloudlinuxSummary._get_average_apps_per_domainc C s | dk s|dk rdS | | S )z�
Get average amount of applications per user
:param total_apps: total amount of applications
:param amount_of_apps_per_user: amount of applications per user
r. Nrr )r� Zamount_of_apps_per_userrr rr rs �_get_average_apps_per_user s z,CloudlinuxSummary._get_average_apps_per_userc C s t dd� | D ��S )z
Get amount of running applications on server
:param apps: list of applications for something selector
c S s g | ]}|j r|j d kr|�qS )Zstarted)Z
app_status)r� �apprr rr rs r) 0 s z@CloudlinuxSummary._get_amount_of_runned_apps.<locals>.<listcomp>)r= )�appsrr rr rs �_get_amount_of_runned_apps( s z,CloudlinuxSummary._get_amount_of_runned_appsc C sB t � }x | D ]}||j d7 < q W |�d�p4dg}|d d S )z�
Get maximum amount of applications per domain
:param apps: list of applications for something selector
r. )r r r )r �doc_root� most_common)r� Zapps_per_domainr� � most_commonsrr rr rs �_get_max_apps_per_domain2 s
z*CloudlinuxSummary._get_max_apps_per_domainc C sB t � }x | D ]}||j d7 < q W |�d�p4dg}|d d S )z~
Get maximum amount of applications per user
:param apps: list of applications for something selector
r. )r r r )r r� r� )r� Z
apps_per_userr� r� rr rr rs �_get_max_apps_per_userC s
z(CloudlinuxSummary._get_max_apps_per_userc C s* t � }x | D ]}||j d7 < q W |S )z�
Get Counter object which contains amount applications per version of interpreter
:param apps: list of applications for something selector
r. )r r5 )r� �apps_per_versionr� rr rr rs � _get_counter_apps_per_versionT s
z/CloudlinuxSummary._get_counter_apps_per_versionc C s& | � |�}|�d�pdg}|d d S )z�
Get maximum amount of applications per version of interpreter
:param apps: list of applications for something selector
r. )r r r )r� r� )rk r� r� r� rr rr rs �_get_max_apps_per_versionb s
z+CloudlinuxSummary._get_max_apps_per_versionc C s( t � }x| D ]}|�|j� q W t|�S )zz
Get amount of domains with applications
:param apps: list of applications for something selector
)rJ �addr� r= )r� r� r� rr rr rs � _get_amount_of_domains_with_appsp s
z2CloudlinuxSummary._get_amount_of_domains_with_appsc C s( t � }x| D ]}|�|j� q W t|�S )zx
Get amount of users with applications
:param apps: list of applications for something selector
)rJ r� r� r= )r� �usersr� rr rr rs � _get_amount_of_users_with_apps~ s
z0CloudlinuxSummary._get_amount_of_users_with_appsc C s | � |�}t|�S )z�
Get amount of applications per each versoin of interpeters
:param apps: list of applications for something selector
)r� r� )rk r� r� rr rr rs �4_get_amount_of_apps_per_each_version_of_interpreters� s
zFCloudlinuxSummary._get_amount_of_apps_per_each_version_of_interpretersc s � fdd�t | �D �S )z�
Get list of versions of interpreters on server
:param interpreters_stats: dict with varied statistics about each version of interpeters
:param state: state of interpeters (installed, enabled)
c s g | ]\}}|� r|�qS rr rr )r� Zinterpreter_stats�stat)�staterr rs r) � s zHCloudlinuxSummary._get_list_versions_of_interperters.<locals>.<listcomp>)r
)�interpreters_statsr� rr )r� rs r� � s z4CloudlinuxSummary._get_list_versions_of_interpertersc C s" t | �}|dk rt|�S t� S dS )zI
Get list of apllications on server for defined selector
N)rV rT )r� Z iter_appsrr rr rs �_get_list_of_applications� s z+CloudlinuxSummary._get_list_of_applicationsc s �x| j D �]
}| j�tt|�dd�|��}|dkrHt| jd| d < q
| j�t| j|�dd| d d�|�� | jjt| j |t
�dd| d d
�|�d d � | jjt| j |t �dd| d
d�|�d d � | j�t| j |�dd�|��� | jj� fdd�dd| d d�|�d d � | jd| d dk| jd| d < | jjt| j
� �dd| d d�|�d d � | j�tt|�dd�|��}|dk �r�|| jd| d < | j�t| j� �dd| d d�|�� | j�t| j� �dd| d d �|�� | j�t| j� �dd| d d �|�� | j�t| j� �dd| d d!�|�� | j�t| j| jd| d | jd| d �dd"�|��}|dk �r�|| jd| d# < | j�t| j| jd| d | jd| d �dd$�|��}|dk �r�|| jd| d% < | j�t| j� �d&d| d' d(�|�� q
W dS ))zc
Fill dict with statistics by varied statistics about ruby/nodejs/python selectors
r` z*Can't get statistics about {} interpretersNZ selector_Z_statusr_ z Can't get status of {} selectorrb Z_versions_installedz7Can't get list of versions of installed {} interpretersF)ro Z_versions_enabledz5Can't get list of versions of enabled {} interpretersz!Can't get list of {} applicationsc s t � �S )N)r= rr )�interpreter_appsrr rs r} � r~ z>CloudlinuxSummary._fill_selectors_statistics.<locals>.<lambda>Z_applications_amountz)Can't get total amount of {} applicationsr Z_usedZ_applications_runningz6Can't get amount of runned application for {} selectorz(Can't get default version of {} selectorZ_default_versionZ _max_applications_per_domainz8Can't get max applications per domain for {} interpreterZ_max_applications_per_userz6Can't get max applications per user for {} interpreterZ_num_users_with_appsz=Can't get amount of users with applications for {} interpeterZ_num_domains_with_appsz?Can't get amount of domains with applications for {} interpeterzFCan't get average amount of applications per domain for {} interpreterZ _average_applications_per_domainzDCan't get average amount of applications per user for {} interpreterZ _average_applications_per_userrd Z _num_applications_by_interpreterzDCan't get amount of applications per each version of {} interpreters)� SELECTORSr� rt r rS r� r� ru r� r� r� r� r� r� rZ r� r� r� r� r� r� r� )rk Zselectorr� Zdefault_version_of_selectorZaverage_apps_per_domainZaverage_apps_per_userrr )r� rs �_fill_selectors_statistics� s�
$
z,CloudlinuxSummary._fill_selectors_statisticsc C s~ ddg} t | dd�\}}}|dkr*t|��t�|�}t� }|d |d< dd� |d D �}x tD ]}|�|t�|d
| < q^W |S ) zA
Get wizard status and list of installed modules
z/usr/sbin/cloudlinux-wizardr� T)r' r �
wizard_statusc S s i | ]}|d |d �qS )r� r; rr )r� r| rr rr rs r� 7 s z<CloudlinuxSummary._get_wizard_statistics.<locals>.<dictcomp>r �wizard_module_)r r r� r, r� rM r� � NOT_SELECTED)r. Zret_codeZstd_outZstd_errZ parsed_json�wizard_statisticsZparsed_modulesr| rr rr rs �_get_wizard_statistics% s
z(CloudlinuxSummary._get_wizard_statisticsc sl | j �| jdd��| j j�fdd�ddddd� x6tD ].� | j j� �fd d�dd
� d �� �dd� q6W d S )
zX
Fill dict with statistics by varied statistics about cloudlinux-wizard
r_ z,Can't get statistics about cloudlinux-wizardc s � d S )Nr� rr rr )r� rr rs r} J r~ z;CloudlinuxSummary._fill_wizard_statistics.<locals>.<lambda>r� z%Can't get status of cloudlinux-wizardF)ro c s �d� S )Nr� rr rr )r| r� rr rs r} R r~ r� z;Can't get statistics about module "{}" of cloudlinux-wizardN)r� rt r� ru rM r� )rk rr )r| r� rs �_fill_wizard_statistics= s"
z)CloudlinuxSummary._fill_wizard_statisticsc C s0 t � } g }x | D ]}|t| | �� �7 }qW |S )zH
Returns list of implemented scripts in integration.ini
)rP rT �keys)r� ZscriptsZsectionrr rr rs �$_get_implemented_integration_scriptsY s
z6CloudlinuxSummary._get_implemented_integration_scriptsc C s2 dg d�}t j�t�s|S d|d< | �� |d< |S )zh
Checks integration script exists and if exists
get list of implemented scripts
F)�integration_scripts_used� integration_scripts_specifiedTr� r� )r� r� r� rO r� )rk rp rr rr rs �_get_integration_infod s
z'CloudlinuxSummary._get_integration_infoc C s8 ddl } d}| �� }t|j�| }t|j�| }||fS )zG
Gets memory usage: total and used memory in megabytes
r Ni )�psutilZvirtual_memoryra �totalZused)r� Z bytes_in_mbZmemZ mem_totalZmem_usedrr rr rs �_get_memory_usedp s z"CloudlinuxSummary._get_memory_usedc C sN t �� } d}d}ttj�rFtj�|�rFt|��}|� � �
� }W dQ R X | |fS )za
Gets kernel info release and module version (starting from 7h)
:return:
z/sys/module/kmodlve/versionN) �platform�releaser r r r� r� r� r� r� r� )�kernel_releaseZkmodlve_version_fileZkmodlve_versionr� rr rr rs �_get_kernel_info| s
z"CloudlinuxSummary._get_kernel_infoc C s
t t� �S )zp
Gets info about lve extensions usage
Calculates amount of packages with lve extensions
)r= rC rr rr rr rs �#_get_lve_extensions_packages_amount� s z5CloudlinuxSummary._get_lve_extensions_packages_amountc C s@ d} t j�| �sdS t| ��}|�� �� }W dQ R X tt|��S )zk
Reads /proc/sys/fs/datacycle/enable in order to check
datacycle enabled parameter
z /proc/sys/fs/datacycle/enableFN)r� r� r� r� r� r� r� r` )Zdatacycle_filer� r� rr rr rs �$_is_kernel_datacycle_enabled_in_file� s
z6CloudlinuxSummary._is_kernel_datacycle_enabled_in_filec C sF d\} }t j�| �sdS t| ��}|�� �� �d�}W dQ R X ||kS )zR
Checks if datacycle parameter was given
for current boot
)z
/proc/cmdlineZ datacycleF� N)r� r� r� r� r� r� �split)Z cmdline_filer� r� r� rr rr rs � _is_datacycle_param_was_passed� s
z0CloudlinuxSummary._is_datacycle_param_was_passedc C s0 t �� } d}x | D ]}|tt �|��7 }qW |S )z=
Returns general amount of domains on server
r )r r> r= r� )Z cpusers_listZ
domains_countr� rr rr rs �_get_total_domains_amount� s
z+CloudlinuxSummary._get_total_domains_amountc C sN t � } tt| �d���}tt| �d���}|r6|r6dS |r>dS |rFdS dS dS )zi
Returns is links traversal protection enabled on server
(symlinks or hardlinks)
z fs.protected_symlinks_createz fs.protected_hardlinks_create�allZ
symlinks_onlyZhardlinks_onlyZnoN)r$ r� r` r� )r� Zsymlink_protection_enabledZhardlink_protection_enabledrr rr rs �%_is_link_traversal_protection_enabled� s z7CloudlinuxSummary._is_link_traversal_protection_enabledc C s@ d} d}d}t j�d�s$t j�d�r(|S t j�d�r8|S | S d S )N�iso�cldeploy�qcow2z/etc/cl-convert-savedz/var/log/cldeployz /etc/cloudlinux-qcow2-install)r� r� r� )r� r� r� rr rr rs �_get_cl_installation_source� s z-CloudlinuxSummary._get_cl_installation_sourcec s �� � � ttj�r`�jj�jddddd� �jj�jdddd d� �jj� fd
d �dd d
dd� td d�s��jjt dddd d� �jjt
ddddd� �jjt ddddd� �j��j ddd� �j��j
ddd� �jj� fdd �ddddd� ��� ��jj�fd d �dd d dd� �jj�fd d �d d!d"dd� ��� ��jj�fd#d �d$d%d&dd� �jj�fd'd �d$d(d)dd� �jj�jdd*d+d d� ttj��r��jj�fd,d �dd-d.d d� �jjtdd/d0d d� �jjtdd1d2d d� �jj�jdd3d4d d� d S )5Nr` Z lve_extension_packages_amountz"Can't get lve extension usage infoF)ro r_ Z!link_traversal_protection_enabledz5Can't get link traversal protection enabled parameterTc s � d S )Nr. rr rr )� kernel_inforr rs r} � r~ z;CloudlinuxSummary._fill_system_statistics.<locals>.<lambda>Zinstalled_kmod_lve_versionz)Can't get installed kmod-lve version info)�skip_jwt_checkZcldiag_cron_check_enabledz&Can't get is cldiag cron check enabledZ
cpu_amountzCan't get cpu amountZtesting_repository_enabledz#Can't get testing repository statusZ base_distroz Can't detect platform for CLOSZsecureboot_statusz+Can't detect whether secure boot is enabledc s � d S )Nr rr rr )r� rr rs r} r~ r� z Can't get kernel release infoc s � d S )Nr� rr rr )�vendor_integration_inforr rs r} & r~ r� z'Can't get integration_scripts_used infoc s � d S )Nr� rr rr )r� rr rs r} - r~ rb r� z-Can't get integrations scripts specified infoc s � d S )Nr rr rr )� memory_usagerr rs r} 6 r~ ra Zmemory_total_mbzCan't get total memory infoc s � d S )Nr. rr rr )r� rr rs r} = r~ Zmemory_used_mbzCan't get used memory infoZ
domains_totalzCan't get domains amountc s � � � p� �� S )N)r� r� rr )rk rr rs r} M s Z kernel_datacycle_usage_enabledz,Can't get kernel datacycle enabled parameterZ virt_typez!Can't get the virtualization typeZhostnamezCan't get the hostnameZ cloudlinux_installation_sourcez Can't get installation source)r� r r r r� ru r� r� r rQ r r r� r� r� r� r� r r
r� )rk rr )r� r� rk r� rs �_fill_system_statistics� s�
z)CloudlinuxSummary._fill_system_statisticsc sB � j �� fdd�ddd� � j �tddd� � j �tdd d
� d S ) Nc s
� j ptS )N)r� �UNKNOWN_RHN_IDrr )rk rr rs r} o r~ zKCloudlinuxSummary._fill_dict_with_rpm_packages_statistics.<locals>.<lambda>r_ r� zCan't get system IDZ
os_versionzCan't get version of OSrf rd z*Can't get info about client's rpm packages)r� ru r rI )rk rr )rk rs r7 m s
z9CloudlinuxSummary._fill_dict_with_rpm_packages_statistics)�paramr{ c C s t �| j�|��S )z�
Retrieve data from proc/mounts for param
:return: param_value - Optional[int],
if there is no value - None
)r' Z
str_to_intr� r� )rk r� rr rr rs �_get_proc_param� s z!CloudlinuxSummary._get_proc_paramc C s� dg}t tj�r.|�dddddddd d
d g
� x8|D ]0}| jjt| j|�d |�d
d�d� |�dd� q4W | jjt
d dddd� dS )z�
Filling stats about mounting
- mount params from parameters list
- separate hidepid getting, since it is more complicated than
other mounting params
zkernel.memcg_oom_disablezfs.enforce_symlinksifownerzfs.symlinkown_gidz fs.protected_symlinks_allow_gidz fs.protected_hardlinks_allow_gidzfs.global_root_enablezfs.proc_can_see_other_uidzfs.proc_super_gidz fs.xfs.cap_res_quota_disablezubc.ubc_oom_disablezfs.process_symlinks_by_taskr` r� r� z
Can't get {0}T)ro ZhidepidzCan't get hidepid valueN) r r r r r� ru r r� r� r� rR )rk Z
parameters�prr rr rs � _fill_proc_params_statistics� s4
z.CloudlinuxSummary._fill_proc_params_statisticsc sZ t dd�� | j�� fdd�ddd� | j�� fdd�dd d
� | j�tt� �dd d � d
S )zv
Fill statistics fields from jwt token.
- jwt token metrics (cl_plus existence and client_id)
F)Zcheck_expirationc s � d kr d S � � dd �S )N�cl_plus)r� rr )r� rr rs r} � s z=CloudlinuxSummary._fill_cln_jwt_information.<locals>.<lambda>r` r� z Can't get cl_plus informationc s � d kr d S � � dd �S )N� client_id)r� rr )r� rr rs r} � s r� zCan't get client_id valueZlicense_activez Can't get license status valueN)rF r� ru r rH )rk rr )r� rs �_fill_cln_jwt_information� s
z+CloudlinuxSummary._fill_cln_jwt_informationc s0 � j �� fdd�ddd� � j �tddd� d S )
za
Filling stats centralized management
- centralized management existence
c s t j�� j�S )N)r� r� r� �CL_PLUS_CM_DISABLED_PATHrr )rk rr rs r} � r~ zKCloudlinuxSummary._fill_centralized_management_statistics.<locals>.<lambda>r` Z centralized_management_disabledz Can't check CM disabling statusr_ Z cl_plus_sender_service_statusz)Can't check cl plus sender service statusN)r� ru rG )rk rr )rk rs �'_fill_centralized_management_statistics� s
z9CloudlinuxSummary._fill_centralized_management_statisticsc
C s� t � } yFt� }x:|D ]2}x,|�� D ] \}}| �d|� �g ��|� q"W qW W n2 ttfk
r~ } zt�d|� W dd}~X Y nX | S )a�
Prepare list of dicts with CPU metrics
Method get_cpu_metrics returns data in following format:
[
{
"id": 0,
"model": "QEMU Virtual CPU version 2.5+"
},
{
"id": 0,
"model": "QEMU Virtual CPU version 2.5+"
}
]
This helper produces a dict, where each key - metric_name,
value - list of values for all CPUs
"cpu_model": [
"QEMU Virtual CPU version 2.5+",
"QEMU Virtual CPU version 2.5+"
],
"cpu_id": [
0,
0
]
Zcpu_z CPU metrics getting error: %sN) r� rJ r< �
setdefaultr� r� rL ri r� )rp Z cpu_coresZcpu_core�metric�valueZexrr rr rs �make_flat_cpu_metrics� s
$ z'CloudlinuxSummary.make_flat_cpu_metricsc C sl ddddg}| � � }x@|D ]8}|dkr*dnd}| j�t|j|d�||d|� d �� qW | j�td
d d � dS )
z�
Filling stats about hardware metrics, specifically:
CPU:
- cache
- frequency
- model
- id
RAM:
- ram
- swap
Zcpu_idZ cpu_cache_mbZ cpu_modelZcpu_frequency_mhzrc rb Nz Can't parse z metric for all coresrd ZmemoryzCan't parse memory metrics)r� r� ru r r� rK )rk Z exp_metricsZ cpu_metricsr� Z metric_typerr rr rs �_fill_hardware_statistics
s
z+CloudlinuxSummary._fill_hardware_statisticsc C s6 | � � \| jd< | jd< | jd< tj�d�| jd< dS )am
Collect CageFS enter errors number:
1. "Act like CageFS is disabled (unable to create LVE).. %d"
2. "Act like CageFS is disabled (unable to enter into NameSpace).. %d"
3. "Act like CageFS is disabled (unable to acquire lock for user %s uid %d)"
4. File-marker /etc/cagefs/fail.on.error presense
:return None
Z(act_cagefs_disabled_unable_to_create_lveZ&act_cagefs_disabled_unable_to_enter_nsZ)act_cagefs_disabled_unable_to_acqure_lockz/etc/cagefs/fail.on.errorZ"act_cagefs_disabled_marker_presentN)�*_scan_log_for_act_cagefs_disabled_messagesr� r� r� r� )rk rr rr rs �$_fill_act_cagefs_disabled_statistics0
s z6CloudlinuxSummary._fill_act_cagefs_disabled_statisticsc C s� y�| � � \}}|dkrv|�d�}ttddd|d��}t|�}ttddd|d��}t|�}ttddd|d��}t|�}n d}d}d}|||fS ttfk
r� dS X d S )
a�
Scan /var/log/messages for all needed "Act like CageFS is disabled ..." messages for yesterday
:return tuple of ints:
Number of "Act like CageFS is disabled (unable to create LVE).. " messages,
Number of "Act like CageFS is disabled (unable to enter into NameSpace).. " messages,
Number of "Act like CageFS is disabled (unable to acquire lock for user %s uid %d)" messages
r �
z2Act like CageFS is disabled (unable to create LVE)T)Z fixed_stringZmultiple_searchZdata_from_filez<Act like CageFS is disabled (unable to enter into NameSpace)z<Act like CageFS is disabled (unable to acquire lock for user)i���i���i���N)�_get_data_from_logr� rT r r= r� r� )rk �
returncode�stdoutZ
lines_listZfound_lines_listZnum_unable_to_create_lveZnum_unable_to_enter_nsZnum_unable_to_acqure_lockrr rr rs r� >
s&
z<CloudlinuxSummary._scan_log_for_act_cagefs_disabled_messagesc C sf t � } | dkrd}n,tj�� tjdd� }|�d�}d|� d�}tj|ddd tjtj d
�}|j
|j fS ) z�
Scan /var/log/messages for all needed "Act like CageFS is disabled ..." messages for yesterday
:return: Tuple (ret code, std_out string)
)Zcl7Zcl7hZcl8za/usr/bin/journalctl --since yesterday --until today | /usr/bin/grep 'Act like CageFS is disabled'r. )Zdaysz%Y-%m-%dz /usr/share/cloudlinux/logs-at z< /var/log/messages | /bin/grep 'Act like CageFS is disabled'Tz /bin/bash)r� r� r� r� �stderr) r �datetimeZdateZtodayZ timedeltaZstrftimer r �PIPEZDEVNULLr� r� )Zos_typeZs_cmdZyesterday_dateZ date_to_scanr� rr rr rs r� ^
s
z$CloudlinuxSummary._get_data_from_logc s� t tj�r � j�� jddd� t� jd< � j�ttj dd�ddd � � j�� j
dd
d � � j�� fd d
�ddd� � j�� fdd
�ddd� t d k r�t d k r�d�
t t �� jd< n
d � jd< � �� d S )Nr` �old_way_of_integration_usedz*Can't detect old LVE integration mechanism�lve_utils_versionF)Z
verify_expr_ Z
cl_editionz Can't get CloudLinux editionZis_solo_meta_file_presentz$Can't detect solo meta file presencec s � j S )N)r� rr )rk rr rs r} �
r~ zACloudlinuxSummary._prepare_cl_normal_statistics.<locals>.<lambda>r� z$Can't get or generate security tokenc s
� j ptS )N)r� r� rr )rk rr rs r} �
r~ r� zCan't get system IDz{}-{}�lvemanager_version)r r r r� ru r� �LVE_UTILS_PKG_VERSIONr r Zget_cl_edition�_is_solo_marker_exists�LVEMANAGER_PKG_VERSION�LVEMANAGER_PKG_RELEASEr� �_fill_dict_with_statistics)rk rr )rk rs r6 u
sD
z/CloudlinuxSummary._prepare_cl_normal_statisticsc s� d�d�d�d�d�t j���s$dS G dd� dt�� � �����fd d
�}�d �d �d i}d d
� t ���D �}|�� xX|D ]P}y|||� W q| � k
r� Y q| tk
r� x|D ] }d||< q�W Y q|X q|W |� | j�< |� | j�< |� | j�< dS )z�
Filling stats lvemanager
- number of visits on X-Ray tab
- number of X-Ray installation
- number of LVEManager opens
z/var/log/cloudlinux/i�Q Zlve_number_of_opensZxray_number_of_installationZxray_number_of_visitsNc @ s e Zd ZdS )zECloudlinuxSummary._fill_lvemanager_statistics.<locals>.BreakOuterLoopN)rv rw rx rr rr rr rs �BreakOuterLoop�
s r� c
s t j �� �� }t�| d�}d}x�|D ]�}y�d�|�d�d d� �}t j �|d��� }|� | k rn|k r�n njd�|�� �d�dd � �}|dkr�| � d7 < q�|dkr�| � d7 < q�|d kr�| � d7 < nd
}W q& tt fk
r� w&Y q&X q&W |�r � �d S ) N�rFr� r� z%Y-%m-%d %H:%M:%S,%fzlvemanager-openedr. zxray-install-startedzxray-tab-clickedT)
r� Znowr� r� r� r� Zstrptimer� r� �AttributeError) Zstats�fileZcurrent_date_timestampr� Zbreak_outer_loop�lineZlog_timeZlog_date_timestampr� )r� �COLLECTING_PERIOD�
LOG_FOLDER�LVE_NUMBER_OF_OPENS�XRAY_NUMBER_OF_INSTALLS�XRAY_NUMBER_OF_VISITSrr rs �fill_statistics�
s(
zFCloudlinuxSummary._fill_lvemanager_statistics.<locals>.fill_statisticsr c S s g | ]}t �d |�r|�qS )z ^lvemanager\.log(\.\d{1,2})?$)�re�match)r� r; rr rr rs r) �
s zACloudlinuxSummary._fill_lvemanager_statistics.<locals>.<listcomp>r )r� r� �isdirrh �listdir�sortr� r� )rk r r� Z log_filesZlog_filer� rr )r� r r r r r rs �_fill_lvemanager_statistics�
s2
z-CloudlinuxSummary._fill_lvemanager_statisticsc s� � j �� fdd�ddd� � j �� fdd�ddd� t� j d < td k rbtd k rbd
�tt�� j d < n
d � j d < � j �� jd d
d� � �� d S )Nc s � j S )N)r� rr )rk rr rs r} �
r~ z7CloudlinuxSummary._prepare_statistics.<locals>.<lambda>r_ r� z$Can't get or generate security tokenc s
� j ptS )N)r� r� rr )rk rr rs r} �
r~ r� zCan't get system IDr� z{}-{}r� r` r� z*Can't detect old LVE integration mechanism)r� ru r� r� r� r� r� r� )rk rr )rk rs �_prepare_statistics�
s,
z%CloudlinuxSummary._prepare_statisticsc C s t tj�r| �� t tj�r,| �� | �� t tj�r>| �� t tj �rP| �
� t tj �rb| � � t tj
�s�t tj�s�t tj�r�| �� tdd�s�| �� t tj�r�| �� t tj�r�| �� | �� | �� | �� | �� | �� | � � | � � | � � | � � | � � d S )NT)r� )!r r ZGOVERNORr9 ZCAGEFSrF r� ZRESELLER_LIMITSrU r rj Z PHP_SELECTORr� �
RUBY_SELECTOR�PYTHON_SELECTOR�NODEJS_SELECTORr� r r� ZLSAPIro ZWIZARDr� �_fill_wpos_statisticsr� rA r� r� r� rq r� r rx )rk rr rr rs r� s>
z,CloudlinuxSummary._fill_dict_with_statisticsc s� | j �| jdd�� t� t�r�� �d�dkr�| j �� fdd�ddd � | j �� fd
d�dd d � | j �� fd d�dd
d � | j �� fdd�ddd � | j �� fdd�ddd � | j �� fdd�ddd � | j �� fdd�ddd� | j �� fdd�ddd� d S )Nr_ z!Can't get AccelerateWP statisticsrp r� c s � d d S )N�
enabled_sitesr� rr rr )�
wpos_statsrr rs r} 2 r~ z9CloudlinuxSummary._fill_wpos_statistics.<locals>.<lambda>r` Z sites_count_with_enabled_wposz*Can't get statistic of using AccelerateWP.c s � d d S )N�
allowed_usersr� rr rr )r rr rs r} 8 r~ Z users_count_with_allowed_wposc s � d d S )Nr � object_cacherr rr )r rr rs r} > r~ Z%sites_count_with_enabled_object_cachec s � d d S )Nr r rr rr )r rr rs r} D r~ Z%users_count_with_allowed_object_cachec s � d d S )Nr �site_optimizationrr rr )r rr rs r} J r~ Z*sites_count_with_enabled_site_optimizationc s � d d S )Nr r rr rr )r rr rs r} P r~ Z*users_count_with_allowed_site_optimizationc s
� � d�S )N� is_accelerate_wp_flag_enabled)r� rr )r rr rs r} V r~ r z1Can't get statistic of AccelerateWP feature flag.c s
� � d�S )N� is_accelerate_wp_icon_enabled)r� rr )r rr rs r} \ r~ r z0Can't get statistic of AccelerateWP icon status.)r� rt �get_wpos_statsr� r� r� ru )rk rr )r rs r ) sZ
z'CloudlinuxSummary._fill_wpos_statisticsc
C sn dg| }yt |dd�\}}}W n. tk
rN } zt�d|� dS d}~X Y nX |dkr\|S t�d||� dS )z�
Run cloudlinux-statistics using subprocess and handle errors.
:type args: list[str]
:rtype: str or None
z /usr/sbin/cloudlinux-statisticsT)r' z.Unable to run cloudlinux-statistics, error: %sNr z;cloudlinux-statistics failed with exit code: %i, output: %s)r r ri r+ r� )r r. r r/ r� r� rr rr rs �_run_cloudlinux_statisticsb s
z,CloudlinuxSummary._run_cloudlinux_statisticsc
C s� | � ddddg�}|dkr dS y t�|�}|d }|d }W n8 tttfk
rt } zt�d t|�� dS d}~X Y nX t |�t |�fS )
zp
Get number of users and resellers with faults for the past 24h
:rtype: tuple[int, int]
z
--by-faultr� z--jsonz --period=1dN)NNrQ r� zISomething really bad happened to cloudlinux-statistics, The reason is: %s)
r r� r, r� r� r� ri r+ r_ r= )rk r/ Z json_datarQ r� r� rr rr rs rS v s
z5CloudlinuxSummary.get_users_and_resellers_with_faultsc C sV |dkr dS |� � }|�d�r"dS |�d�r0dS |�d�r>dS |�� rJdS d| S dS )zGet config cpu limit formatN�%�speedZmhzZghzZold_cpu_formatz unknown: %s)�lower�endswith�isdigit)�clsr^ Zunitrr rr rs r_ � s
z&CloudlinuxSummary._get_cpu_limit_unitsc C s | dkr dS d| d S )z/Convert memory limit from mempages to megabytesNr1 i rr )r� rr rr rs r` � s z!CloudlinuxSummary._mempages_to_mbc C s8 | dkr dS t j| |pdd�}|dkr*dS t|d d�S )z,Convert cpu and ncpu to percents of one coreNr )Zlncpug Y@r. )r
Zconvert_to_kernel_format�round)r^ rW r rr rr rs r] � s z(CloudlinuxSummary._cpu_limit_to_percentsc C s� | j dkr|ttdd� |j�� D ��td�d�}g }x6t|td�d�D ]"\}}tt|��}|�||f� qDW |j dd� || _ | j S )zC
Return list of tuples [users_in_package, package]
Nc s s | ]}|j jd k r|V qd S )N)�packager; )r� r� rr rr rs r� � s z>CloudlinuxSummary.get_users_amount_per_plan.<locals>.<genexpr>r! )�keyT)�reverse)
r� �sortedrT r� r� r r r= r� r
)rk rX Zusers_sorted_by_packager� r! �groupZ num_usersrr rr rs �get_users_amount_per_plan� s
z+CloudlinuxSummary.get_users_amount_per_planc C s, y| � |�|d S tk
r& d S X d S )Nr. )r& �
IndexError)rk Z
number_of_toprX rr rr rs rh � s z5CloudlinuxSummary._get_top_package_by_number_of_usersc C s" t |�\}}|s | �t|�� |S )z%
Parse CLI arguments
)r/ r� r) )rk r r� r� rr rr rs r � s z CloudlinuxSummary._parse_argsr� r c C s8 | t � � |d�}|dk r ||d< t|d� t�|� dS )z?
Print data in default format for web and exit
)rp r� r� NZstatistic_enabledT)r� r( �sysr� )rp r� Z exit_coder8 r� rr rr rs r� � s
z(CloudlinuxSummary._print_result_and_exitr. c C s* | � dt�� i� t| d� t�|� dS )z�
Print error and exit
:param dict message: Dictionary with keys "result" as string and optional "context" as dict
r� TN)r� r� r( r( r� )r� r� rr rr rs r� � s
z!CloudlinuxSummary._error_and_exitc C s6 t j�d�r2tjddgdtjtjd�} t�| j�S dS )zD
Return mod_lsapi info from switch_mod_lsapi script
z/usr/bin/switch_mod_lsapiz--statT)r� r� r� N) r� r� r� r r r� r� r, r� )r� rr rr rs rn � s
z$CloudlinuxSummary.get_raw_lsapi_infoc C sd d} t � r| d7 } tj�| �r`tj| dgdtjtjd�}t�|j ��
d�}dd� |� � D �}|S d S )
zd
Return wmt-api config
if is_solo_edition True, run wpt-api-solo is command
z-/usr/share/web-monitoring-tool/wmtbin/wmt-apiz-soloz --config-getT)r� r� r� r� c S s i | ]\}}t |�|�qS rr )r_ )r� �kr� rr rr rs r� s z9CloudlinuxSummary._get_wmt_api_config.<locals>.<dictcomp>N) r r� r� r� r r r� r� r, r� r� r< )Zcommandr� Zwmt_api_config_summaryrr rr rs rp s z%CloudlinuxSummary._get_wmt_api_configc C s t j�d�S )Nz /etc/cloudlinux-edition-solo)r� r� r� rr rr rr rs r� s z(CloudlinuxSummary._is_solo_marker_existsc C s0 t j�d�r,tjddgddd�} t�| j�S dS )zL
Return statistics from cloudlinux-ssa-manager get-stat API
z /usr/sbin/cloudlinux-ssa-managerzget-statT)r� �capture_outputN)r� r� r� r r r� r, r� )r� rr rr rs r� s
z CloudlinuxSummary.get_ssa_statsc C s: d} t j�| �r6tj| dgddd�}t�|j��d�S dS )zE
Return statistics from cl-smart-advice counters API
z/usr/sbin/cl-smart-adviceZcountersT)r� r* r� N) r� r� r� r r r� r, r� r� )�utilr� rr rr rs �get_smart_advice_stats% s z(CloudlinuxSummary.get_smart_advice_statsc
C s~ t ddd�}tdd�sz| j�| jdd�}|dk rzyt |d |d d�S tk
rx } zt�d t|�|� |S d}~X Y nX |S )
z=
Construct dict with Smart Advice statistics
N)Zsmart_advice_totalZsmart_advice_appliedT)Zskip_marker_checkre z!Can't get Smart Advice statisticsr� ZappliedzAUnexpected json response from server, field %s is missing in "%s") r� r r� rt r, r� ri rj r_ )rk rp Zsa_statsr� rr rr rs r� 3 s&
z.CloudlinuxSummary._get_smart_advice_statisticsc C s4 d} t j�| �r0tj| dgddd�}t�|j�S dS )zE
Return statistics from cl-smart-advice counters API
z/usr/bin/clwpos-adminzget-statT)r� r* N)r� r� r� r r r� r, r� )r+ r� rr rr rs r S s z CloudlinuxSummary.get_wpos_stats)r� Nr N)r. )�rv rw rx r� r� � itertools�compressr r r r r
r r� r� r r� r� � staticmethodr� �propertyr� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r r r0 r5 r� r r r r` r7 r$ r r r% r r� r8 r9 rA r@ rE rF rO rP rR rU rc re ri rj ro rq rx rw r_ r4 ry r6 r rv r3 r� r� r r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r7 r1 r� r� r� r� r� r� r� r� r� r6 r r r� r r rS � classmethodr_ r` r] r& rh r r� r� rn rp r� r� r, r� r rr rr rr rs r� � s
7 + (
- " DAj
T: (%
#'LR
v
- &! ,F $9
r� c C s t � } | j�� | S )N)r- rK Z
link_xml_node)r� rr rr rs r� b s
r� )�Z
__future__r r r- r �builtinsr r� r� r r* r� r� r� r( r� r� r� r � functoolsr r � collectionsr r r Z future.utilsr
�operatorr Zmultiprocessingr Zsocketr
Zpathlibr r� Z cldetectlibr� r
Zclcommonr Zclcommon.constr Z clcommon.libr Zclcommon.lib.cleditionr r r r r Zclcommon.cpapir Zclcommon.utilsr r r r r r r r r Zclcommon.clwpos_libr! r" r# Zclcommon.sysctlr$ Zclconfigr% r& r' Z cli_utilsr( r) Zcllimitslib_v2r* r+ Z
clveconfigr, Zlveapir- Zcl_summary_arg_parser/ �typingr0 r1 r2 r3 r4 r5 r6 r7 Zclsummary.cl_summary_utilsr8 r9 r: r; r< r= r> r? r@ rA rB rC rD rE rF rG rH Z!clsummary.rpm_packages_statisticsrI Z clsummary.hardware_statisticsrJ rK rL Zclwizard.modulesrM Z lve_utilsrN r� Zvendors_api.configrO rP Z cldiaglibrQ Zcl_proc_hidepidrR Zclselect.clselectstatisticsrS rT rU rV rW rX Zclselector.selectorlibrY Zclselect.clselectctlrZ Z
lvemanagerr� r[ r� � ImportErrorr� r ri r� r� r� rC r� r� r� ZERRORr� r^ r� r� �objectr� r� rr rr rr rs �<module> s� , (L
R