Dolphin-7.x Resource Hog FLASH xml member_queries browse.php

First of all this has been addressed and basically ignored --> http://www.boonex.com/unity/blog/entry/xml_php_resource_hog_confirmed_

it has not gone away, if anything it has gotten worse.

i think code satori mentioned there were a few areas where the xml.php file is located on the dolphin platform. the issue i am seeing is coming from basically two locations /flash/xml.php /modules/boonex/xml.php (might be wrong on this one, im tired, been staring at processes run for the past hour. the images will tell the story.

I monitored two servers, both are shared hosting environments with ~20 accounts hosted on each one. of those accounts, there are ~7 of those sites that have more than 5 members. there are two accounts that have in excess of 3k members, these are for testing purposes and are not jacked stolen, ripped off with the infamous mods that are sold here for that purpose.

To start with the site(s) images i am depicting have been edited to protect the account owner, i have blottered out the account name for their protection. the first site i am talking about here, has exactly 18 members, and none were online when the RESOURCE SPIKES were taking place.

what is the requirement here for the /modules/index.php to hit this type of CPU spike?

here it is again from another account and from same server. except this one shows the /flash/XML.php file acting up as well.

seriously what could the /modules/index.php need with 71% CPU? note the mysql process was included so you could see which machine this was coming from.

as you can see this is from a different server, and just watch how crazy this gets.32% really isnt that bad, and it really is only for a burst. but there is something causing this, and its not site activity, there is something in the application that needs to be looked at. why would a menu query on a site with no activity spike to 32% CPU. this just doesnt make sense. now this site really gets crazy, and i just dont understand this whole ordeal. this has been discussed and those of us who spent our time exploring and investigating were essentially ignored. that could prove to be the case on this go-round as well, but i sure hope not. there are accounts being suspended from the hosts who are listed on the recommend hosting page, irony, accident, or by design. so you grab a hosting package for $4.95/month two months later, you are told  you are suspended for excessive resource usage, and will need to upgrade to VPS. hmmmm, i will not elaborate on this because that is not what this is about to me at this moment, but you have all read the posts in the forums where members were suspended and told they would need to upgrade to VPS.

/flash/XML.php --> /modules/index.php --> /member_menu_queries.php. resource usage not all that bad  you might say on this one. not really, but again, there is nobody on this site, there is no activity on the site. so what is causing this?

without running an actual timer on these occurrences, i would guess that the time between spikes is about 1 minute. same files spiking the CPU.

now this spike is not something i see often, but thought it interesting the CPU spike with no users on this account at the time of the spike. logically how many people would you think would need to hit this blog at one time, in order to  knock off this type of resource? i dont have any idea, but these are not issues that can be managed from the server-side. this is directly related to the source code.

and i dont have a clue what the heck happened here. browse.php seriously guys, what would cause this, it certainly isnt traffic.

this is just an idea of what resources are used on the cron jobs. and if setup based on the required suggestion from the developers, to run every minute on the minute, this is heavily resource intensive. what is causing the CPU usage on these cron jobs? since they are running from the database, shouldnt they be more optimized than this? how come this only started with dolphin 7, this was not an issue with Dolphin 6?

View the Images offsite

this is not a bug report, this is not a complaint, this is a blog about what i see going on with the Dolphin 7 platform. there was an instance where we were told to turn off the profiler, then told to turn on the profiler. which is it, and what is causing these CPU spikes?

hopefully we can get some resolution. AlexT or any other developer who wishes to have access to my servers, please inquire, you can certainly take a look. i would also like to see the processes  that are running on the demo-dolphin site you guys run, because there were times when it was pegged out i am sure, and you have a great deal of members on that site. but a site with 18 members, should not be spiking the CPU +20% muchless the astonomical amount that you see depicted in these images.

Regards,

DosDawg

HoustonLively, i had to edit my post in order to provide a definitive answer.  since you are not allowed to add images to the post comments.

HL as soon as i read your inquiry, i went to check on the status. The above is what was being used. Then i just sat there and watched and it took ~15 minutes, but the results were astonishing:

so as you can see, even though you killed all instances of the IM, there is still something running the /flash/XML.php file, and its still spiking the CPU. Weird thing is this is ~76 lines of code, and using 70% of the CPU.

I have this strong suspicion that the ilnes 17 - 22 have something peculiar to do with this CPU spike. Since the security mechanism was broken upon release, and nothing has been done to fix it, we were only told to disable it, and that they would get to it at a later date.

define('BX_SECURITY_EXCEPTIONS', true);
$aBxSecurityExceptions = array(
'POST.data',
'REQUEST.data',
);

now to see the entire /flash/XML.php file

<?
/***************************************************************************
*
* IMPORTANT: This is a commercial product made by BoonEx Ltd. and cannot be modified for other than personal usage.
* This product cannot be redistributed for free or a fee without written permission from BoonEx Ltd.
* This notice may not be removed from the source code.
*
***************************************************************************/
$sModule = isset($_REQUEST['module']) ? $_REQUEST['module'] : "";
$sAction = isset($_REQUEST['action']) ? $_REQUEST['action'] : "";

if(($sModule == "mp3" && $sAction == "screenshot")
|| ($sModule == "board" && $sAction == "transmit")
|| ($sModule == "games" && $sAction == "makeShot")
|| ($sModule == "livecam" && $sAction == "channelShot")
|| ($sModule == "photo" && $sAction == "transmit"))
{
define('BX_SECURITY_EXCEPTIONS', true);
$aBxSecurityExceptions = array(
'POST.data',
'REQUEST.data',
);
}

$sGlobalHeader = "modules/global/inc/header.inc.php";
if(!file_exists($sGlobalHeader)){ header("Location:install/index.php"); exit;}

require_once('../inc/header.inc.php');
require_once($sGlobalHeader);
require_once($sIncPath . "constants.inc.php");
require_once($sIncPath . "db.inc.php");
require_once($sIncPath . "xml.inc.php");
require_once($sIncPath . "functions.inc.php");
require_once($sIncPath . "apiFunctions.inc.php");
require_once($sIncPath . "customFunctions.inc.php");

$sModule = empty($sModule) || !secureCheckWidgetName($sModule) ? GLOBAL_MODULE : $sModule;

$sContents = "";
$sContentsType = CONTENTS_TYPE_XML;

if($sModule == GLOBAL_MODULE)
{
require_once($sIncPath . "xmlTemplates.inc.php");
require_once($sIncPath . "actions.inc.php");
}
else
{
$sModuleIncPath = $sModulesPath . $sModule . "/inc/";
require_once($sModuleIncPath . "header.inc.php");
require_once($sModuleIncPath . "constants.inc.php");
require_once($sModuleIncPath . "xmlTemplates.inc.php");
require_once($sModuleIncPath . "customFunctions.inc.php");
require_once($sModuleIncPath . "functions.inc.php");
require_once($sModuleIncPath . "actions.inc.php");
}

switch($sContentsType)
{
case CONTENTS_TYPE_XML:
//--- Print Results in XML Format ---//
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
header ('Content-Type: application/xml');
$sContents = "<?xml version='1.0' encoding='UTF-8'?>" . makeGroup($sContents);
break;
case CONTENTS_TYPE_SWF:
header("Content-Type: application/x-shockwave-flash");
break;
default:
break;
}
echo $sContents;
$oDb->disconnect();
?>

then looking at this file, /flash/XML.php i see where it calls /modules/global/inc/header.inc.php and this file has what looks like a bunch of garbage code in it. why would you need to define the database parameters again for this file, and what is this exactly doing, and why. there is no user parameters, and there are no actual database queries here, so why would you need to have all of the database parameters here again, when  you have them in the /inc/header.inc.php file? why would we just not require /inc/header.inc.php in this instance.

/**
* Data Base Settings
*/
if(!defined("DB_HOST")) define("DB_HOST", $db['host']);
if(!defined("DB_PORT")) define("DB_PORT", $db['port']);
if(!defined("DB_SOCKET")) define("DB_SOCKET", $db['sock']);
if(!defined("DB_NAME")) define("DB_NAME", $db['db']);
if(!defined("DB_USER")) define("DB_USER", $db['user']);
if(!defined("DB_PASSWORD")) define("DB_PASSWORD", $db['passwd']);
if(!defined("DB_PREFIX")) define("DB_PREFIX", "Ray");
if(!defined("GLOBAL_MODULE")) define("GLOBAL_MODULE", "global");
if(!defined("MODULE_DB_PREFIX")) define("MODULE_DB_PREFIX", DB_PREFIX . ucfirst($sModule));

does that appear to be a bunch of garbage that is essentially useless?

there's some more of the story. i do see that the ones who should be reading this are avoiding this post. interesting to say the least. i think i will go invite the powers to come take a looksee.