Data Usage Thoughts, Tools, Tactics

[LWSChad]

Hi All,

I had a great conversation with Caspio support a few weeks ago that helped my significantly reduce the amount of data that my app uses.

 

Here are the cliff notes.

1. To see how much data your pages use, paste the URL from the deploy menu into a new web browser, and use a development tool or web inspector to investigate. (in Chrome press F12, and click the Network tab)
2. Records per page is a big factor in data usage... nearly a linear relationship.
3. Ajax is a big load up front, but will help if you encounter many "paging" actions... long reports.

Happy Building

[AndreiaBrazier]

thanks for sharing

[Elena]

Good to know. I've received a warning a couple of weeks ago that my data usage is nearing my max, and that I will be charged accordingly.

 

I know I have to edit some of the queries in some of my tabular pages. A few of my tables take a bit to load because of the queries.

 

But does the size of the view table or source table have a causal link to this too?

[skauntey]

Quite useful thread on the topic!

 

Adding further to the discussion, I observed that once embedded the data transfer is nearly 2/3 times higher than individual data page. And in some cases (charts for example) this goes up to nearly Ten folds.  Perhaps some could explain why is this happening?

 

Besides I found this announcement made in Nov 2015; may not be relevant in this context but thought I would link it any way.

 

http://blog.caspio.com/caspio-news/update-on-caspios-data-transfer-policy/

 

Best

[LWSChad]

 

But does the size of the view table or source table have a causal link to this too?

 

From my understand of what support told me, no.

[LWSChad]

Adding further to the discussion, I observed that once embedded the data transfer is nearly 2/3 times higher than individual data page. And in some cases (charts for example) this goes up to nearly Ten folds.  Perhaps some could explain why is this happening?

 

 

I think the external file at https://b5.caspio.com/scripts/e1.js contributes.

 

Even though that's small it has to look to a library that would help Caspio build the tables.

The URL is straight off of their servers and seemingly could serve some things directly.

 

** I don't know at all.  Just applying what I do understand to this scenario.

[aam82]

Just chiming in that I've had extensive discussion with support and my findings were the same.

[DataCobalt]

Has anyone tried taking the Caspio styling and deploying that on your actual web page and removing it entirely from the Caspio side? As in set the style to a completely blank one so it doesn't load with each datapage? 

[svitlanataran]

If you want to have all the styles in one place as in your external CSS file you need to define the styles within your CSS file using the embedded HTML tags. Example:

To define font-family for Caspio label you should look for the element tag on the web page (In Firefox you can install Firebug add-on to easily see the page elements without going into the source), so after you find the element tag which in the label case is "label" then you can write this code in your CSS:

#caspioform label{
font-family:Arial;
}

So basically .cbFormLabel class is not recognizable in your CSS file. This class is defined in Caspio Style object so when you embed a DataPage using this Style object, Caspio server compiles and converts these styles into inline styles and deliver it to the web page browser. That is why if you remove these classes within the Caspio Style, you need to define your own styles in the CSS for each Caspio element as explained in the example above.

 

And I don't think, this will reduce data transfer.

[aam82]

Has anyone tried taking the Caspio styling and deploying that on your actual web page and removing it entirely from the Caspio side? As in set the style to a completely blank one so it doesn't load with each datapage? 

 

I was able to reduce data usage by dp.asp by 2KB by copying Caspian, zeroing it out, and loading it locally instead. It can't truly be zeroed out, even in souce mode, but you can remove all parameters from each selector so that you have just table{} etc. in your Caspio CSS. Not a big improvement.

 

However..

 

I just ran a test, and found that my JS embed datapage draws 711KB for just 81 records, and the same datapage in IFRAME or URL deployment draws 125KB! (The report has a couple of unoptmized SQL queries. There might be more savings here. I think Caspio does its own optimization behind the scenes; nothing I did to my queries, views, etc, made the page any lighter.) I asked support to add this to the documentation this morning.

 

So now I'm working on using IFRAMES where possible, but they're much harder to style!

[DataCobalt]

 

 

So now I'm working on using IFRAMES where possible, but they're much harder to style!

 

 

Also be careful as they make parameters MUCH harder to pass and receive. You can wind up having page data accidentally cross over into other iframes if you work with multiple tabs.

[qing22]

 It can't truly be zeroed out, even in souce mode, but you can remove all parameters from each selector so that you have just table{} etc. in your Caspio CSS. Not a big improvement.

[jhanajc]

Shocked to find my Data Transfer usage has somehow exceeded 1GB in just 2 weeks! 

 

How this is possible in view of the data transfer policy? http://blog.caspio.com/caspio-news/update-on-caspios-data-transfer-policy/ when there are no live users on the system, only me developing it!

 

I will query this with support on Monday but it is potentially a show stopper for me because I can't charge back these levels of over-usage fees to customers.. 

 

Seriously, if it's not a mistake and can't be resolved I'll have to pull the plug immediately and use another platform... lol 

[MayMusic]

When you open a page from browser, import, export data to Caspio it goes toward your data transfer usage. You can use developer tools like firebug to see how much a page usages for each load. It is under Net tab and look for Size.

 

You can try to contact your assigned success manager and see if they can wave the developing period over-usage fee for you.

[JEllington]

In  one of his videos Ned Pajic explained what the different modules in the deploy code do and pointed out the ones that do not  need to deployed. I have since not been able to find this video. Nonetheless I think with the new CB upgrade, the deploy modules have changed anyway. Are there any modules in the deploy code that can be left out to help reduce bandwidth overhead?
I was charged for going over the 1GB data transfer limit and had to up to 2GB. I have not even put the site into production yet. Caspio needs to revisit the imposed data transfer limits.

[Farnsbarnes]

Wow, I've only just started developing my application and you guys are scaring me.

I hadn't realised Caspio was so hungry on its data transfer.    My marketing strategy will be to cast the net wide and generate high usage on a free part of my web database, but only a small proportion of clients can be expected to upgrade to a paid service.   This is a very common business model in a wide range of sectors.   Yet It's easy to see Caspio's charges running into 10's of thousands of dollars, killing my business before I get the chance to build a revenue generating client base.

 

I've no problem paying those kinds of fees once they're affordable.  But if Caspio is going to strangle the small guys before they have a chance to grow, its marketing should be clear that they are not a solution for the business start-ups.

I think I need to look around!

[jhanajc]

This could invalidate my business model. 

 

Because I've already spent so much time in Caspio, I will see what happens to data stats through optimization:

 

1. Disabling Ajax on all pages.

2. Streamlining page data as much as possible.

3. Any images optimized to the max.

 

However, a major part of my design is document uploading.. and I haven't even started testing this yet!  so I might need to consider an external file repository.

 

Another problem is report printing, going to look at setting up an external PDF generator using PHP/API calls.

 

Caspio are certainly not making it easy for the small guys on a wing and a prayer with little budget. (this is the reality) and if people such as us are squeezed out through startup costs, they will be missing out on a few of these small ventures becoming much larger ventures.

[Farnsbarnes]

Well guys, it seems that Caspio has been listening to our concerns regarding the cost of data transfer.  By pure chance I've just revisited the pricing section of the website and discovered that their pricing plans and content have been updated.  It appears that Caspio is sensibly giving much more help to the smaller guys, while loading the higher fees on to the larger companies.

 

Data transfer is now "unmetered" from the professional plan upwards.  I'm assuming that "unmetered" means 'free', although I'm not sure why they don't just say "free".  I hope it's just semantics and that I'm being over-suspicious!

And a quick glance shows that pivot tables are at last included in the professional plan.  On the downside the professional plan still doesn't include pdf downloads (why not??!!), and the number of datapages is restricted to 50.

 

But overall the changes to data transfer bring a huge sigh of relief to business start-ups like me.  It also means I can be less paranoid about measuring specific cost efficiencies of my datapages and use that time on buidling my business instead.

[DataCobalt]

I actually just had a discussion with our account rep regarding this.

 

Despite it still listing data caps on the pricing page it is truly unmetered with the new pricing structure.

 

I clarified that if by chance my application got popular, and I blew past their expexted usage amounts, what would happen then?

 

They told me that I would not be forced to purchase a higher plan and no data transfer fees would happen, it is truly unmetered.

[JEllington]

This is a concern for me as well, I too called my Caspio rep on this..
Apparently it depends on the plan you are subscribed to. From what I was told the "Professional Plan" does offer unlimited data transfers but you are limited to 50 datapages. The "Business Plan" (what I currently use) has a 1GB transfer limit with and additional charge for going over, last march I was charged $80 for going 500MB over the allotted 1GB. I have since added an additional 1GB to my plan at $50 a month. Both the Professional and Business plans are $250 a month but offer different package limits.
What you need to plan for is if you will be needing more that 50 datapages a month. I have already used 96 of the 100 allotted datapages in my plan and will have to increase them soon at a rate of $25 for 5 additional datapages. 
Curious to see my actual data transfer usage when the site goes live next month.

SideNote: as aam82 points out in the above post IFRAME or URL deployment will save considerably on transfer rates as the code is run on the CB side. The downside (for me at least) is refreshing parameters outside the iframe and styling.
Its funny or sad depending on the point of view, that in normal HTML practices iframes are frowned upon and blacklisted as things to never use. but within Caspio we struggle to use them to perfection. 

[aam82]

Spend a few days away from the forum and all hell breaks loose! Well, the new plan works better for us and them. We upgraded, but we're getting more, and not pinching pennies with deployment hacks that were making for an uneven UX. Looks good to me.

[jhanajc]

Really pleased with the new pricing plans, thank you Caspio!!!!

[scribblemilk]

I'm on the old Business plan at $249 for 1GB usage and 100 datapages, I like a lot of you are still in development and I regulary hit my usage limit, this is an extreme worry to me especially after I have spent thousands of dollars with caspio developing the site.

Once the site is live it will take time to build a sufficient online community willing to pay for services, until they do the Performance plan with unlimited metering is simply too expensive at $799 and the Professional plan simply to small with 50 datapages, the 1GB usage upgrades cost $50 each which based on my calculations isn't cost effective as even paying users wouldn't cover that cost, likewise adding datapages to the Professional plan equally isn't cost effective.

I would say the gap between the Professional and Performance pricing plans is simply too big for start ups, they could easily fail before they begin. With hosted cloud servers charging cents per 1GB of usage to be charged $50 per GB looks like opportunism to me and also short sighted as it will stunt growth.

I have looked at ways of reducing usage, and there are reductions that I can make but they are all small reductions across the board and not enough in the long run, this is extremely disappointing and I have written to my Customer Success Manager to discuss, but the more of us that post about our concerns the better I would say.

[JolliBeng]

Hi everyone!

Just to update this post, Caspio now has a new set  of plans. You can now have unmetered DataTransfer on Explore and higher plans. Explore plan costs $29. 

You can see here the other plans and pricing. 

 

-JolliBeng

[AtayBalunbalunan]

There has been changes in the data transfer feature of the Explore plan. It now has only 500 MB data transfer per month. Unmetered data transfer starts now from Build. For more information, please see here.

On 11/13/2018 at 6:53 AM, JolliBeng said:

Hi everyone!

Just to update this post, Caspio now has a new set  of plans. You can now have unmetered DataTransfer on Explore and higher plans. Explore plan costs $29. 

You can see here the other plans and pricing. 

 

-JolliBeng

[NailDyanC]

Hi, just to update this post, Caspio now has a new set  of plans. These new plans offer different storage limit and also unmetered Data Transfer in all plans including free plan.  You may check this link: https://www.caspio.com/pricing/