This page as a pdf file

CGMS accuracy

Nanette Freedman wore MiniMed's glucose monitor several times. She looks at how accurate the results were

Like most other diabetics, I feel the frustrations of trying to follow recommendations to balance insulin, food and exercise to achieve normal blood glucose levels. I try my best in order to feel well and lead a normal active life today, and also at the back of my mind lurks that spectre of complications in the future, and linked to it the hope that good blood glucose control now could maybe, if I'm lucky, keep those complications at bay. However, despite my best efforts, my blood glucose fluctuates frequently, shoots up and down without warning, and disappoints me again and again just when I think I have finally identified and corrected some source of the problem. Part of the problem of course is lack of information, and part (but only part!) of the information that is lacking concerns blood glucose. I only know my blood glucose when I stick my finger and measure it, somewhere from 4-10 times a day - the other 1430 minutes out of the 1440 minutes of the 24 hour day, I rely on my best guesses, and these may sometimes be way off. So the opportunity to try out a Continuous Glucose Monitoring System, even in the limited form that it is currently available, sounded very attractive indeed.

Picture of MiniMed Continuous Glucose Monitoring System

From the start I was aware of certain limitations of the system - the Minimed Continuous Glucose Monitoring System (CGMS) can only operate for about 72 hours at a time, and worse, there is no possibility of reading the results as you go along, rather all the data is stored in the monitor, downloaded to a PC at the end of the 72 hours, and then calibrated and analyzed. In addition, at least 4 blood glucose measurements every day obtained in the normal way from a glucometer have to be entered into the monitor to enable it to perform the calibration at the end. 4 blood glucose measurements a day does not sound very much for many of us, but a day for this purpose is rigidly defined as midnight to midnight, so 4 measurements have to be provided for the partial first and last days of the 3 day test, as well as for the two middle days which are really 24 hours long. Careful records must be kept concerning food intake, insulin doses, and physical activity during the 72 hour test. The little sensor of the monitor is inserted with an automatic insertion device. Insertion was not a problem (close to painless), but I was disturbed to discover that unlike the set of an insulin pump, apparently a huge amount of tape was required to ensure that the sensor remains in place. I sometimes have allergies to tape, but in reality this was not bad, on two of the 3 three-day tests I had slight itchiness from the tape towards the end, and only on the third did I have more severe itchiness with skin blistering - by my standards that is not too bad. Also the sensor cannot be disconnected for showering, so I was provided with sort of plastic bags to hang the sensor round my neck for showers.

All this did not contribute to the fun of the experience, but my feeling was that I would willingly suffer all these minor inconveniences and more, if in the end, the CGMS data could then be used to give accurate assessment of the body's response to food, insulin doses and exercise, so that modifications could be made.

In a Commentary article (The FDA Panel Advises Approval of the First Continuous Glucose Sensor, BH Ginsberg, Diabetes Technology & Therapeutics 1,2,1999,203-4) Barry Ginsberg writes, concerning the Minimed Continuous Glucose Monitoring System, "it was clear that the device was not as accurate as self monitoring of blood glucose". As I used this device and saw the results it generated, my attention shifted from irritation at not being able to see and make use of the information about my blood glucose here and now while using it, and focussed rather on questions about the accuracy of the device.

Why doubt the accuracy of the device? partly some of the things which were staring me in the face when I looked at the tables and graphs resulting from my first test of the CGMS. And partly, my education and work experience came into play here: I do not have any professional training relating to diabetes management, but most of the approximately 30 years of my working life so far has been spent looking at and analyzing physical measurements of biological data (I work as a Medical Physicist in the field of Nuclear Medicine), so it has become almost second nature to me to perform at least some reality check on data which comes my way.

These are the things which troubled me first:

discrepancy between glucometer and CGMS readings
at least one CGMS hypo when I really did not feel hypo - and note that I have good hypo awareness except within the first few hours after a previous hypo
at least one hypo which I experienced as a hypo and measured on the glucometer as a hypo, but the CGMS assessed as within normal limits
mismatch, sometimes quite severe, between plots at midnight
correlations presented in the table, are:
1. based on too few points to be meaningful, and
2. correlation sounds impressive, but may not be the most relevant information
correlations and mismatch generally deteriorated over the 3 days of the tests in my case

All this was very disappointing, and raised for me the following questions:

are the CGMS and glucometers measuring the same thing?
how reliable are measurements made by the CGMS in daily use?
how is the CGMS device calibrated?

In an attempt to understand these issues better, I went to the literature, starting with the Minimed CGMS web page, which gives some interesting information, but falls short, in my opinion, by stating that there are references to the device in the scientific literature, but failing to specify them. I performed a Medline search to access references to continuous glucose monitoring, and read whatever I could lay hands on. References to those papers I quote from are listed, and I would be happy to pass on, on request, a more complete list of the references I found. Below I will try to summarize what I learnt from the literature, regarding the methodology used for the CGMS, and its implications about the accuracy of the CGMS. I was a little disappointed to find that papers describing the Minimed CGMS were almost all written by people who are associated with Minimed, rather than by detached physicians and scientists. I am also a bit sceptical about the relevance of short trials of any device for assisting in blood glucose control, since in my personal experience, I find that the relatively intense care and attention from physicians and paramedical personnel associated with any new or special trial or regime often has a remarkable short-term effect on my blood glucose, regardless of what the new regime is - I have attributed it to some combination of enhanced self-control in food choices and timing of eating, positive attitude and state of mind, and the increased physical activity which, for me at least, generally accompanies such optimism and interest.

CGMS measures glucose in interstitial space, while glucometers measure either plasma or whole blood glucose. Studies found in the scientific literature mostly (but not unanimously) indicate that these two glucose concentrations, while not identical, will be very close except perhaps under certain circumstances, e.g. when glucose levels are changing very rapidly, when interstitial glucose may lag behind blood glucose. For the device as it stands now, this does not sound as if it should be a major problem, since the same trends would be measured in interstitial fluid and blood. If measurements in interstitial fluid were to be used in some future system to warn the patient of rapidly developing hypoglycaemia, the short delay in the event of fast dropping glucose levels could be a disadvantage.

If the CGMS were not using a reasonably reliable monitoring method, it would not have got as far as it has. However, the literature describes only one or two careful trials checking the reliability in daily use, i.e. normally clothed persons wearing the CGMS for 72 hours of active life. One obvious study I would have liked to see is a study of subjects wearing more than one CGMS - do the devices all say the same thing? I participated in a small study wearing two devices for 3 days - one of them, unbeknownst to me, packed up half way though, but for the duration that they were both functioning, they agreed well for some but not all the time. The two devices I wore were inserted close to one another (left lower abdomen) so presumably were subjected to the approximately the same knocks and bumps in the course of my daily life - I did wonder, but do not know, if discrepancy might have been worse had they been placed in different locations. What I gathered from my own data and that of others in this 'two monitor' study worried me somewhat. It was not a surprise that values were not identical on the two devices, but there were instances, when the trace was ascending on one and descending on the other.

Calibration of the CGMS is done by looking separately at every 24 hour period. Unfortunately the system does not have the flexibility to start the 24 hour period from whatever hour of the day you start using the sensor, but rather from midnight to midnight. This means, as I mentioned above, that, for example, if you start using the sensor at 3 pm, you have to perform and enter at least 4 glucometer measurements between 3 pm and midnight that day. Similarly on the last day, you have to enter at least 4 measurements between midnight the previous night and the hour at which you stop using the sensor. If the sensor packs up before the end of 72 hours (happened to me) and you haven't yet performed 4 glucometer readings that day, then it will not be able to calibrate that day's results.

Suppose all went smoothly, and you succeeded in measuring and entering sufficient glucometer readings, then for each 24 hour midnight-to-midnight time period, the data is calibrated separately. The 4 (or more) glucometer readings are paired with the corresponding sensor readings, and 'modified linear regression' is performed. Linear regression is a statistical technique which estimates the straight line which will come closest to fitting all your data points. The assumption behind linear regression is that the two variables, in this case the glucometer reading and the CGMS reading, are linearly related to each other, i.e. that there really is a straight line which will go through all the data points. There are two issues here which beg questions - firstly can this assumption be true if performance of the sensor is not constant with time (and it is presumably concerns that the performance may not be constant with time which lead Minimed to calibrate each 24 hour period separately, and which determine that the sensor should only be used for 3 days), and secondly even if there are only statistical variations, and change in performance of the sensor in a 24 hour period is negligible, then are 4 points sufficient to give an accurate linear fit. It has been indicated to me that Minimed actually use an additional point, forcing the line to go through the origin or some other point, so there are effectively 5 points, but this is still very few points for accurate determination of a straight line. Think about it - if you only had 2 points, the two would exactly fit to a straight line regardless of how well any other points would fit the line, 5 points is not much more than 2. 5 points is also too few points on which to base a meaningful calculation of a correlation coefficient. In addition, the accuracy of the straight line fit will depend on the spread of the points. Intuitively, one would guess that if all the points you measure are within some very small range - say 3.9-5.0 mmol/L (70-90 mg/dL) - then one might feel very confident that about the calibration for other parts of the curve which are within or close to this range, but have very little confidence in calibration values which appear to be around 17 mmol/L (300 mg/dL) or down to 1.7 mmol/L (30 mg/dL). Don't think it totally hypothetical that one could only have glucometer readings between 3.9 and 5.0mmol/L (70 and 90 mg/dL) to enter - just occasionally, when conducting careful monitoring, my blood glucose temporarily behaves extraordinarily well - one of my test days was probably the only day since diagnosis of DM that every time I measured blood glucose it was between 3.9 and 5.0 mmol/L (70 and 90 mg/dL) - I even ate a chocolate biscuit with the intention of forcing it up in order to see how good or bad a job the CGMS was doing, but failed to catch a peak, though according to the CGMS there was a very delayed peak late that evening when I was asleep - I don't doubt that there was a peak, but given that there were no calibration points even close to it, I doubt its magnitude - 22.2 mmol/L (400 mg/dL)!.

The results reported in the literature for the CGMS sound good, but closer examination raises real doubts here too. In a study of 62 patients who wore 5-10 sensors sequentially (Efficacy and Reliability of the Continuous Glucose Monitoring System, Todd M.Gross, John J.Mastrototaro, Diabetes Technology & Therapeutics 2,S1,2000,S-19-26), and were asked to perform at least 11 glucometer readings a day (4 readings to be used for calibration, the remainder for assessment of accuracy), mean discrepancy was small, -0.3 mmol/L (-5.4 mg/dL) but standard deviation was 2.5 mmol/L (44.2 mg/dL). Calculation of these statistics assumes that the discrepancies belong to a normal distribution, and in a normal distribution, 70% of readings lie within mean +/- 1 standard deviation, with 95% lying within 2 standard deviations. Translating this into numbers in our present case, we see that one would expect almost one in three of the sensor readings to disagree with the 'true' glucometer reading by more than 2.5 mmol/L (44.2 mg/dL) and, one in 20 of them to stray further from the true reading by more than 4.9 mmol/L (88.4 mg/dL). This has serious implications. Particularly if you look at the curves, these curves are relatively smooth, not very spiky - if they were spiky I would conclude that those outlying values were the spikes, but since they are smooth, I have to be concerned that whole sections of the curves may be some way off. For example, I suspect that my false positive and false negative hypos may have resulted from these problems.

So, did I gain anything in terms of personal diabetes management, from my tests of the CGMS - yes, definitely - one very concrete piece of information I discovered was that every day after breakfast (consisting of cereal with plenty of semi-skimmed milk) my blood glucose went shooting up into the stratosphere. This did not occur after my other meals which have a higher fat and protein content and/or lower carbohydrate. Evidently despite the wholegrain high fibre content of my careful choice of cereals, this meal was digested very fast, and taking my bolus of insulin (Humalog) 15-30 minutes prior to eating rather than with the meal has reduced this problem substantially - the second test of the CGMS verified this. I have no doubts that the CGMS can be useful to many people in identifying unfortunate patterns based on incorrect assumptions like this one.

In conclusion, the CGMS can be very useful. However there is a problem. It is very easy to be tempted by the attractive and clear tabulated and graphical presentation to accept every detail of the results. The warnings are there, but they are in small print, and their implications may not be clear to all. If you have tested this device, were you and your physician aware that the FDA Panel advised that '..values of glucose produced by the CGMS should not be used to make therapeutic decisions, but rather should be used to ... provide indications of the best time to do Self Monitoring of Blood Glucose (SMBG). Actual changes in insulin regimen should be based upon the values from the SMBG'? In reality CGMS data may be accepted implicitly by trusting patients and physicians, who have not read all the small print, and do not necessarily have sufficient time and training to performing the careful reality checks which are still warranted by the current state of the device. It is hard to believe that the device always identifies accurately the timing and more importantly the severity of every episode of hyper and hypoglycaemia; and it is worrying to think that decisions of patient management might be based on this.

Author: Nanette Freedman <nanette_f@yahoo.com>. Nanette Freedman is not a medical professional in diabetes care. She has Type 1 diabetes and uses an insulin pump. The information given here is based on her own personal experience. More about Nanette Freedman...

Created: September 2001; Last updated: Monday 1 October 2001

Other pages about blood testing

Reader comments

On 29 Dec 2001 Louise wrote:

Thanks Nanette, for a clear and real explanation of your experiences. This is the 'referenced' view I've been after (am in health care also). Do you think this system would be 'useful' in giving an indication (even just an inkling) of what goes on from 10pm til 8am, as I have peaks every morning and can't establish why, despite all my best efforts/night tests etc. Any thoughts?
- On 8 Jan 2002 Nanette replied:
  
  Louise, thanks for your appreciative comments.
  
  I do think the CGMS could be very useful in understanding what goes on during the night, but in the light of what I read, would recommend that you use it only as a guide, and do not trust the exact values. That is, it seems to me reasonable to believe that a dip in blood glucose as shown by the CGMS is indeed a dip, but having identified such a pattern (if it is indeed a repeating pattern and not a once off phenomenon), I would then want to set an alarm (also on more than one night) to wake up and test with a glucometer at whatever hour the dip occurs, in order to determine if it were really hypoglycaemia or just in the low normal range, and if so how severe it is. In other words, I regard the CGMS in its current state as a very useful tool for identifying trouble spots, but not as a device which is sufficiently accurate to define them completely. Also be aware of the phenomenon of unusual behaviour during test periods.