Do Not Jump Ship - Borneman Salt Study

[Spracklcat]

Hi all,

I had the chance to talk to Eric at length after MACNA about the test and here are some answers to some of the questions that ahve been posed:

1. Yes, 10 tanks, one for each salt brand. 10 "batches" of each salt were tested , one per month by emptying the tank at the end of the month and refilling it with SW mixed from the next "batch". Interesting to note that most salts don't include lot numbers, so we have no idea if the salts were mixed at different times from ingredients from different suppliers. No way to know since there's no regulations.

2. There were no skimmers on the tanks.

3. statistics still need to be crunched. Kim and Eric are well aware of the possible degrees of freedom here.

4. Different salt batches were tested by doing near-100% water changes once monthly with the new batch. (Problem here of course with incomplete WC, and no way to tell whether changes in tank were immediate and caused by new batch or long-term caused by old batch).

5. Tanks were covered with acrylic tops to cut down on contamination from room air.

6. Only one replication has been done as of yet (unless you consider each batch of salt a replication, though I don't like that--). This was a costly and time-consuming test. More replicates will wait for more $$.

7. I don't think the focus of the study was to measure salt quality by algae growth--the algae/cyano results were incidental. The goal was to see which salt did the best job at promoting coral growth and overall health. Measuring nutrient depletion would not have addressed this. Bare tanks with chemical nitrogen source would not have addressed this either, although that would be a good experiment to do just to isolate the algae promotion. But in real life, the salt addition is such a minor factor in bioload in tanks compared to uneaten food, waste, etc that I wouldn't bother.

8. Autoclaved CC was used to eliminate bacteria in the sand as a possible nutrient processor/effector. Clearly bacteria coming in on the surface of corals/fish/macro couldn't be controlled for except if they all came from the same water source, and even then it's dodgy.

It is an ambitious study and a good start. And even better, good to see people doing their own "peer review" to analyze the results. I am in agreement with solbby as I usually am (hi there): MOre needs to be done and we need to find a quantitative way to analyze the data. IMO, that will be the toughest part. I am assuming they are going to use growth in coral weight or some such. How can you quantify algae promotion other than by eye at this stage? And doing it taht way, how can you measure whether that algae promotion is statistically significantly different between tanks?

So, for myself I would use the results of the study (WHEN they are released--as they really haven't been yet, no?) to weigh which salt is best for growth--NOT which salt is best for keeping algae at bay. I give them tremendous credit for even starting the discussiona nd doing the massive amount of work it took, and I hope it stimulates others to attempt further study.

Christine

 

[cali_reef]

Hi Christine, you were too busy every time I walked by you.

Love your new magazine, I didn't know my good buddy Scott Fellman is writing for you guys until he told me to buy the magazine.

 

[jhale]

Nice posting Spraclcat.

I have read the articles Eric wrote, not all the threads jim linked.

One quick question, if we all use some kind of buffering system, CA reactor, 2 part, etc, does it really matter what salt is used ?
This is assuming you do not get a bad batch of salt with overly high or low readings. I know trace elements are important, but those need to be supplemented as well. just an observation.
still, kudos to running the experiment.

 

[cali_reef]

From the first post of this RC thread.

I did not hear him say that in the presentation and I did not fall a sleep in this one. When I asked him afterwords he said "the data still needs more reviewing".

I am switching to it based on the few data points that were shown as part of the presentation and my past experience with two buckets of RC on two different occasions. RC is not the same as IO when I tested the newly mixed water, and the $3-5 extra dollar is probably worth it.

 

[ShaunW]

Hi Christine, WELCOME TO MANHATTANREEFS. My reefing home! Glad you can join us here.

Great post, all studies have pilot experiments, that lead the researchers down the right path towards further enlightenment and sometimes the incidental/unexpected results are the most exciting.

 

[ShaunW]

So, for myself I would use the results of the study (WHEN they are released--as they really haven't been yet, no?) to weigh which salt is best for growth--NOT which salt is best for keeping algae at bay. I give them tremendous credit for even starting the discussiona nd doing the massive amount of work it took, and I hope it stimulates others to attempt further study.

Christine

The thing is pandora's box is open. We all battle against algae at some point. If certain salts do promote algae growth more than others, that is important to know.

 

[Spracklcat]

Thanks for the warm welcome all! (from us "out-easters" )

Pierce, you should have just pushed your way to the front of the booth and DEMANDED a magazine-- I would have given it to you. I did go looking for you but didn't see you after that one time. And yes, I was swamped busy promoting the magazine. I hardly got out to see the rest of the trade show.

For me, I'm sticking with IO (for my home tanks and lab tanks)--don't have a strong preference one way or another, but in the end I think barring bad salt batches, it's all the other stuff we put in our tanks that makes bigger differences. I've heard that Red Sea is indeed a good quality salt, but some time ago they had issues with producing a consistent product. To me, that is worse-- and unless the industry demands an accounting of the ingredients and quality control of the product, each bag or bucket is a mystery and you hope for the best.

Good to see you too Solbby, my lab rat friend

 

[prattreef]

How come I had to pay???? j/k The mag. looks very nice and I agree with your analysis of the salt situation. Randy

BTW- Welcome to MR!!!!

 

[Spracklcat]

You had to pay, well, cause otherwise I don't eat.

Do read carefully and let me know if you like/dislike things, any ideas etc so we can evolve and have it include things people would be interested in.

And no making fun of my silly editor's pic.

 

[cali_reef]

OT.. I am glad you guys decided not to have Scott's and Bob's picture in the articles.:lol2:

 

[Chiefmcfuz]

What magazine?

 

[cali_reef]

C... the Journal (new Reef Magazine!)
http://www.readingtrees.com/books_in_print.htm

 

[tomzpc]

Calling it a magazine is almost an insult. The first issue is very beautifully done. It's almost akin to being National Geographic for reefers. =)

Oh...still using IO, for now.

 

[Spracklcat]

Thanks Tom and Pierce! Yes, we were going for the Nt. Geographic feel with a twist of fun (see the Sock Puppet Chronicles). I hope you all like it (and subscribe!)

We did think about putting authors pics in, but we didn't have many, and worse you can imagine what pics we have of Bob .

But from the seahorse.org party at MACNA we have plenty of pics of aquarists doing unusual things so...you never know

 

[pecan2phat]

Thanks for the warm welcome all! (from us "out-easters" )

Pierce, you should have just pushed your way to the front of the booth and DEMANDED a magazine-- I would have given it to you. I did go looking for you but didn't see you after that one time. And yes, I was swamped busy promoting the magazine. I hardly got out to see the rest of the trade show.

For me, I'm sticking with IO (for my home tanks and lab tanks)--don't have a strong preference one way or another, but in the end I think barring bad salt batches, it's all the other stuff we put in our tanks that makes bigger differences. I've heard that Red Sea is indeed a good quality salt, but some time ago they had issues with producing a consistent product. To me, that is worse-- and unless the industry demands an accounting of the ingredients and quality control of the product, each bag or bucket is a mystery and you hope for the best.

Good to see you too Solbby, my lab rat friend

Interesting Christine

I myself, am banning IO from any of my reef tanks. I did a amateur test with my reef tanks at home. I was using SC Reef Salt for about 6 months and my alk, ca, mg stayed consistent at 8.8 dkh, 420 ppm, 1250 ppm with a ca reactor running.
I decided to do a temporary substitute with IO since there was a controversy on RC regarding the amount of borate in the SC salt and I performed 8 weekly 10% water changes of 22g with IO salt. When I tested my levels on the 9th week, this is what I saw after doing a re-test:
Alk-6.1 dkh, Ca-385 ppm, Mg-1150
This is for a 9 week period with eight 22g water changes with IO salt and also with the Ca reactor running at the same exact rate.

IMO, IO is not worth using for an sps dominated reef tank unless you plan to dose heavily to compensate for it's short comings.

 

[nyfireman3097]

WELCOME CHRISTINE
Pecan i had basically the same thing happen to me i changed up due to IO bein the only salt avail when i was desprate annd it threw everything outta wack on me

 

[jhale]

you might want to test and buffer the IO salt mix before you put it in the tank. It's not made for reefers the way sea chem reef salt is, or some of the other mixes. and even SC is not perfect. I had problems with their salt being too high in CA, one bucket tested at 500CA.
At least with IO you know the levels will be the same, even if you have to correct them

 

[Spracklcat]

Thanks guys--

All the little experiments we do with our own tanks I think are really valuable. I think too that especailly with long-running tanks, they do become their own individual systems, and what works for one person's little universe in glass may or may not be the best for another. That said, new setups and newbie reefers really do benefit from all of you old-timers trying things out and being able to say "try this" or "don't use that--it's bad".

I wonder too--how much of trying a new salt and having it do poorly is from the quality or suitability of the salt, and how much is from the creatures being accustomed and adapted to the old salt and not liking the change in their environment?? Does anyone have any experiences with changing to a new salt and having their tank get much better?

 

[sihaya]

Hello all... great thread you have here. Did you all know it's one of the top results when you type in "salt study borneman lowe" into Google. Congrats

Here's my take on the salt study thread (note, it's becoming increasingly hard to find do to aggressive censorship measures--j/k sorta). Though, please, before I say anything, let me insist that I'm not at all trying to attack Mr. Borneman or Ms. Lowe. I'm ONLY questioning the merits of this study. So here it goes...

Basics first:

First, let's talk a little about experiment design so you all can understand what's going on here:

The most basic kind of experiment design is where you look at one dependent variable responding/correlating to one independent variable. For example, let's pretend you're doing a study that looks at muscle growth (dependent variable) and steroid use (independent variable). You could take a 100 mice, inject half with steroids and half with saline solution, give them the same exercise routine and diet for 3 months, then measure their muscle mass at the end. You have to use a lot of mice for both the experimental group getting the steroids and the control group getting saline solution in order to minimize error due to differences among individual mice. To understand this more clearly... suppose you had only used two mice, one control and one getting steroids. At the end of the 3 months, you wouldn't be able to "trust" the results because you can't be sure that the mouse who got the steroids wasn't at a genetic advantage for muscle growth. I think everyone gets this basic idea, right?

Moving on...

So, what do you do when you don't have 100 mice? What if you only have two mice? Can you still do the study? Perhaps. You might be able to do a repeated measures study. What the heck is a repeated measures study? Glad you asked...

One of the most popular and well known repeated measures design is the pretest, posttest experimental design. For example, you can take the two mice, measure their muscle mass at the very start of the study, then weekly for 4 weeks. Then you inject both with saline solution and continue your measurements for another 4 weeks. Next you inject both with steroids and repeat weekly measurements for yet another 4 weeks. Because you're not comparing results of two different mice, but results over time at intervals on the same mice, you gain statistical power. Get it? Think about it for a sec, you will.

This of course, is not the only example of a repeated measures design. There are all kinds of these study designs. But the basic idea is the same... to test the individuals with different "treatments" over time. You tend to do this when you don't have enough subjects to separate into study groups as you would in a "normal" experiment.

Now, finally, about the salt study:

We have 10g tanks, one for each salt plus a control of natural sea water. We have one independent variable and multiple dependent variables measured over time. Now, first off, what kind of study does this look like? Does it look more like the first example I gave of having 100 mice or the second of having 2 mice? It kinda looks like a mix of both, right? Let's take a deeper look...

Statistically and conceptually, it looks very much like a classic experimental design flawed by having only one subject per variance of the independent variable (i.e. one tank per salt).
Note: "We show that there can be extreme variation among identical tanks, even without any live animals" - Toonen and Wee (http://www.advancedaquarist.com/2005/7/aafeature)

Mr. Borneman, however, would like us to think of this as being more like a kind of repeated measures study to be analyzed with ANOVA (a mathematical concept/model used to analyze this kind of data). Even being most generous with the boundaries of logic and reason, I could only accept this claim if the salt brands were consistent. But they are not. Again, as Mr. Borneman himself concedes, the salt brands are often inconsistent even between batches. So, even with all the power and forgiveness one can gain from a repeated measures study, it doesn't apply here because the batches of the sand brands weren't consistent and experimenters only made this inconsistency more pronounced by doing 100% water changes with each new batch of salt.

Now for how this study could have been done (in light of the statistical power afforded to some repeated measure study designs):

Instead of studying one salt in one tank, they should have studied all the salts in all the tanks... over time. For example, the experimenters could have started with natural sea water until the tanks were "cycled." Then every 3-4 months, changed the salt brand in all the tanks until all the tanks had seen all the salts for a period of 3-4 months (taking measurements of dependent variables at time intervals all along the way and with each change of salt brand). Granted, there are a lot of salt brands to test, so this could take a long time. However, they could have also split the tanks into groups of 5 and tested half the salts on 5 tanks and the other half of the salts on the other 5. Then they could have halved the time to do this kind of study.

The downfall of this proposed idea, and the problem with many repeated measures studies, is that the subjects can "fatigue" or "learn" over time. In the example given with the mice, the mice may have "bulked" up by the time they got the steroids, therefore perhaps limiting the additional effect the steroids might have. In this case, the tanks would be experiencing the salts at different ages... and that would be a problem. However, that would be a statistically manageable problem since all the tanks would be aging at the same time.

Ok, I could have more I could say, but this is a long post already. And I think I've made my point. I'm not being "close-minded" and my objections are not "non-sensical"... nor am I trying to embarrass/offend the experimenters. I'm simply looking at this study with a critical eye and right now it looks worthless.

 

[sihaya]

In response to an astute comment made by a fellow reefer:

Your ideas for a better exp. design are good, with one potential problem: the order you change the salts. Going from brand A to B to C may have completely different effects than C to B to A. or not. you'd have to test it to know for sure...

Right, right, of course... I think that's called "recall bias" maybe? I'm not sure of the most appropriate term for the kind of bias you're talking about. But I do know what you're talking about. I thought of that while I wrote this, but I was too lazy/tired to go on about it. But, yeah, theoretically, you'd have to do ABC, ACB, BAC, BCA, CAB and CBA... and that could get to be a headache real fast.

But actually, after a little thought, I'd think the best route would be to do every combination of pairs of salts (two for each, AB and BA for example). To see why I say this... think about this, the number of possible sequences for X salts is X!, while the number of possible nonrepetitive pairs for X salts is (X^2) - X or X(X-1), even multiplying that by two, 2X(X-1) is still considerably less than X!. See what I mean?

------------
Regarding the 100% water changes...

Interestingly, the rationale given for this seems to contradict itself. Borneman et al reason that the 100% changes are done BECAUSE the salts are inconsistent batch to batch. But that seems like funny reasoning to me. If the salts are inconsistant from batch to batch, then why would you want to ensure that the full effect, (even beyond that felt by "normal" home aquariums which almost never do 100% changes), is felt by the tanks?

Besides that, as a practical matter, you can't even do a truly complete 100% water change without completely drying everything in the tank, including the sandbed.

And you see, this is where the weakness of the design becomes almost humorous to me. They have a set of treatments that vary (i.e. different brands of salt). But then within each "treatment," each application (each batch) is different. So it's almost like this matrix of varying treatments such that you don't really just have 11 treatments applied 10 times, but actually 110 treatments each applied once. And yet, this is somehow supposed to ultimately make for a "repeated measures study?" Come on...seriously? lol ::sigh::

In brief, we need not be wasting our time trying to study salt brands as such. What we need to be studying are compositions of salt mixes! Because the salts brands themselves vary from batch to batch, it makes it pretty impossible to do any kind of worthwhile study on them.