diff options
author | 2014-07-01 22:14:42 +0100 | |
---|---|---|
committer | 2014-07-08 16:27:29 -0700 | |
commit | 30c760c1e68b49e11a22ea0febe161e5eeb49530 (patch) | |
tree | 6ba7f207209cfb0d108e410b9a4a53059c0ec904 /drivers/staging/comedi/drivers/usbduxsigma.c | |
parent | staging: comedi: allow COMEDI_BUFINFO, COMEDI_CANCEL and COMEDI_POLL on locked subdevice (diff) | |
download | linux-dev-30c760c1e68b49e11a22ea0febe161e5eeb49530.tar.xz linux-dev-30c760c1e68b49e11a22ea0febe161e5eeb49530.zip |
staging: comedi: usbduxsigma: fixing rounding error in the range struct
The AD converter on the usbsigma turns vref (2.5V/2) into the
binary value 0x780000. However the full scale range goes up to
0x800000 so that vref/2 needs to be upscaled by 0x800000/0x780000
which is 1.06bar. In the driver it's been 1.06 without the infinite
6666. I've put the fraction in the code so that the compiler can
calculate the most precise scaling value by itself.
Signed-off-by: Bernd Porr <mail@berndporr.me.uk>
Reviewed-by: Ian Abbott <abbotti@mev.co.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Diffstat (limited to 'drivers/staging/comedi/drivers/usbduxsigma.c')
-rw-r--r-- | drivers/staging/comedi/drivers/usbduxsigma.c | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/drivers/staging/comedi/drivers/usbduxsigma.c b/drivers/staging/comedi/drivers/usbduxsigma.c index ccc3ef7ba55c..e869af4784c6 100644 --- a/drivers/staging/comedi/drivers/usbduxsigma.c +++ b/drivers/staging/comedi/drivers/usbduxsigma.c @@ -136,7 +136,7 @@ static const struct comedi_lrange usbduxsigma_ai_range = { 1, { - BIP_RANGE(2.65 / 2.0) + BIP_RANGE(2.5 * 0x800000 / 0x780000 / 2.0) } }; |