Qbasicnews.com
April 12, 2021, 07:51:56 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Back to Qbasicnews.com | QB Online Help | FAQ | Chat | All Basic Code | QB Knowledge Base
 
   Home   Help Search Login Register  
Pages: 1 [2]
  Print  
Author Topic: Array Not Dimensioned Problem  (Read 11713 times)
vspickelen
Member
*
Posts: 26



WWW
« Reply #15 on: December 30, 2005, 11:52:18 AM »

C'mon Torahteen,
you certainly don't consider yourself a "bad programmer,"
suspicious perhaps, but probably just running out of patience.

I promise: you're so close to having a working program,
t'would be a pity if I'd finish in your place.
That final satisfaction will be all yours...

(and I wanna see my guidance bear fruit.)

I uploaded an .exe here,
http://home.graffiti.net/vspickelen:graffiti.net/doos/torah10.zip
about 200 training cycles will suffice. Looks nice eh?

Now tell me what problems still remain after my previous post.
vspickelen
Logged
Torahteen
Ancient Guru
****
Posts: 744



« Reply #16 on: December 30, 2005, 12:30:54 PM »

Why is it that it keeps returning values above 1? The sigmoid functio should never return a value above 1.
Logged

quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
vspickelen
Member
*
Posts: 26



WWW
« Reply #17 on: December 30, 2005, 02:01:54 PM »

Quote from: "Torahteen"
Why is it that it keeps returning values above 1?


Quote from: "Deleter"
none of those results are above 1 :Huh:


I didn't notice:
Computer programs use so called exponential notation for expressing large and small numbers, e.g.
 0.04305563 = 4.305563 * 10^-2 = 4.305563e-002

see?
vspickelen
Logged
Torahteen
Ancient Guru
****
Posts: 744



« Reply #18 on: December 30, 2005, 07:36:51 PM »

Oh. Silly me :oops:. Anyway, I'm getting an Illegal Operation error when I run my latest version of code. I'm probably not understanding something. It appears to be happening in the DoNet function. Here is my code (I'm making this page really big lol)

[syntax="qbasic"]DEFINT A-Z

Declare Function sig(x As Double) As Double
Declare Sub DoNet()
Declare Sub TrainNet()
Declare Sub RunNet()
Declare Sub InitNet()

Const IMG_WIDTH = 12
Const IMG_HEIGHT = 12
Const NUM_IL = IMG_WIDTH * IMG_HEIGHT                'Number of Nodes in Input Layer
Const NUM_HL = 20                                    'Number of Nodes in Hidden Layer
Const NUM_OL = 10
Const LR = 0.1
Const GAIN = 5

Type Node
    wt(1 To NUM_IL) As Double        'Weights
    act As Double                       'Output of this Node
    th As Double                        'Threshold
    e As Double                         'Error
End Type

Dim Shared hl(1 To NUM_HL) As Node             'Hidden Layer
Dim Shared ol(1 To NUM_OL) As Node             'Output Layer
   
'Digit Images
Dim Shared img(0 To 9,1 To 12, 1 To 12) As Byte
Dim Shared curImg(1 To IMG_TOTAL) As Byte
'Begin Main
'Load the Images
For i = 0 To 9
    For y = 1 To 12
        For x = 1 To 12
            Read img(i,x,y)
        Next x
    Next y
Next i


InitNet
Print "First, see how the NN performs before training"
RunNet
Sleep
Cls
Print "Now training network"
TrainNet
Cls
Print "Now see how the NN performs after training"
RunNet
Sleep
End

SUB DoNet()
    DIM sum AS single
   
    'Run through nodes in HL
    FOR i = 1 TO NUM_HL
        sum = 0
        FOR j = 1 TO IMG_TOTAL
            sum = sum + hl(i).wt(j) * curImg(j)
        NEXT j
        hl(i).act = sig(sum - hl(i).th)
    NEXT i
   
    'Run through nodes in OL
    FOR i = 1 TO NUM_OL
        sum = 0
        FOR j = 1 TO NUM_HL
            sum = sum + ol(i).wt(j) * hl(j).act
        NEXT j
        ol(i).act = sig(sum - ol(i).th)
    NEXT i
END SUB

Sub InitNet()
    Randomize Timer
   
    'Hidden Layer
    For i = 1 To NUM_HL
        For j = 1 To NUM_IL
            hl(i).wt(j) = (Rnd - .5) * .1
        Next j
    Next i
   
    'Output Layer
    For i = 1 To NUM_OL
        For j = 1 To NUM_HL
            ol(i).wt(j) = (Rnd - .5) * .1
        Next j
    Next i
End Sub

Sub TrainNet()
    Dim numCycles As Integer
    Dim current As Integer
    Dim j As Integer
    Dim sum As Double
   
    Input "How many training cycles", numCycles
    For i = 1 To numCycles
        j = 1
       
        current = Int(Rnd(1) * 10)
           
            For y = 1 To IMG_WIDTH
                For x = 1 To IMG_HEIGHT
                    curImg(j) = img(current, x, y)
                    j = j + 1
                Next x
            Next y
           
        DoNet
       
        'Calculate OL Error
        For j = 1 To NUM_OL
            If j = current Then
                ol(j).e = (1 - ol(j).act) * ol(j).act * (1 - ol(j).act)
            Else
                ol(j).e = (0 - ol(j).act) * ol(j).act * (1 - ol(j).act)
            End If
        Next j
       
        'Calculate HL Error
       
        FOR j = 1 TO NUM_HL
            sum = 0
            FOR k = 1 TO NUM_OL
                sum = sum + ol(k).e * ol(k).wt(j)
                hl(j).e = hl(j).act * (1 - hl(j).act) * sum
            NEXT k
        NEXT j
       
        'Adjust Weights
        FOR j = 1 TO NUM_OL
            FOR k = 1 TO NUM_HL
                ol(j).wt(k) = ol(j).wt(k) + LR * ol(j).e * hl(k).act
                ol(j).th = ol(j).th + LR * ol(j).e
            NEXT k
        NEXT j
       
        FOR j = 1 TO NUM_HL
            FOR k = 1 TO IMG_TOTAL
                hl(j).wt(k) = hl(j).wt(k) + LR * hl(j).e * curImg(k)
                hl(j).th = hl(j).th + LR * hl(j).e
            NEXT k
        NEXT j

       
        Print ".";
    Next i
End Sub

Sub RunNet()
    Dim quit As Byte
    Dim num As Byte
    Dim j As Integer
   
    j = 1
   
    Input "Which number do you want to test?", num
   
    For y = 1 To IMG_WIDTH
        For x = 1 To IMG_HEIGHT
            curImg(j) = img(num, x, y)
            j = j + 1
        Next x
    Next y
               
    DoNet
   
    For i = 1 To NUM_OL
        Print "Value of output node ";
        Print i - 1
        Print ol(i).act
    Next i
End Sub

Function sig(x As Double) As Double
    sig = 1/(1 + Exp(-GAIN * x))
End Function[/syntax]

I left out the DATA statements.
Logged

quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
speedlemon
I hold this place together
*****
Posts: 874



« Reply #19 on: December 30, 2005, 08:55:51 PM »

It's throwing that error because IMG_Total has not been declared.:wink: (i think).
Logged
Torahteen
Ancient Guru
****
Posts: 744



« Reply #20 on: December 30, 2005, 11:15:18 PM »

Great, now FBIde is causing an illegal error everytime I try to compile it. Man, what is my problem  :roll:.
Logged

quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
Anonymous
Guest
« Reply #21 on: December 31, 2005, 02:00:43 AM »

when all else fails compile from command line
Logged
vspickelen
Member
*
Posts: 26



WWW
« Reply #22 on: December 31, 2005, 10:16:04 AM »

Quote from: "Torahteen"
Man, what is my problem  :roll:.

Perhaps being a slovenly reader :wink:?


Since this is Your learning experience: i'll try again.
Code:
   'Calculate HL Error
       
        FOR j = 1 TO NUM_HL
            sum = 0
            FOR k = 1 TO NUM_OL
                sum = sum + ol(k).e * ol(k).wt(j)
                hl(j).e = hl(j).act * (1 - hl(j).act) * sum
            NEXT k
        NEXT j
What's the "outer-loop expression" here, and why is it misplaced?
(This isn't about neural networks intricacies - t'is a very basic error.
Once you see it, you'll never make that mistake again.)

Great find in sub RunNet: j = 1
This one belongs in the same innocent category:
Code:
       current = INT(RND * NUM_OL)
Code:
       'Calculate OL Error
        FOR j = 1 TO NUM_OL
            IF j = current THEN


But all of this will be of no avail until you choose to remember that
Quote from: "I"
This GAIN also enters the derivatives of the sig-function.
(triggers the obvious question... )

vspickelen
(Aka The Truly Annoying Fart, who holds the believe that one should at least have digested his own code.)
Logged
vspickelen
Member
*
Posts: 26



WWW
« Reply #23 on: January 04, 2006, 10:52:50 AM »

Three days on and back to work again tomorrow, so it's time to cast off
this amusing thread, which is indeed grossly overstretched already.

All that's left now, is to present the working code you've all been waiting for.
Note how minute the changes!

[syntax="FreeBasic"]DEFINT A-Z
'****************************************************************
'***  Torahteen's backpropagation / propelled by vspickelen  ****
'URL's: http://richardbowles.tripod.com/neural/neuron/neural3.htm
' & http://richardbowles.tripod.com/neural/backprop/backprop.htm
'****************************************************************
DECLARE FUNCTION sig(x AS DOUBLE) AS DOUBLE
DECLARE SUB DoNet()
DECLARE SUB TrainNet()
DECLARE SUB RunNet()
DECLARE SUB InitNet()

CONST IMG_WIDTH = 12
CONST IMG_HEIGHT = 12
CONST NUM_IL = IMG_WIDTH * IMG_HEIGHT   'Number of Nodes in Input Layer
CONST NUM_HL = 20                       'Number of Nodes in Hidden Layer
CONST NUM_OL = 10
CONST LR = 0.1
CONST GAIN = 5

TYPE Node
    wt(1 TO NUM_IL) AS DOUBLE        'Weights
    act AS DOUBLE                       'Output of this Node
    th AS DOUBLE                        'Threshold
    e AS DOUBLE                         'Error
END TYPE

DIM SHARED hl(1 TO NUM_HL) AS Node             'Hidden Layer
DIM SHARED ol(1 TO NUM_OL) AS Node             'Output Layer
   
'Digit Images
DIM SHARED img(0 TO 9,1 TO 12, 1 TO 12) AS Byte
DIM SHARED curImg(1 TO NUM_IL) AS Byte
'Begin Main
'Load the Images
FOR i = 0 TO 9
    FOR y = 1 TO 12
        FOR x = 1 TO 12
            READ img(i,x,y)
        NEXT x
    NEXT y
NEXT i


InitNet
PRINT "First, see how the NN performs before training"
RunNet
SLEEP
CLS
PRINT "Now training network"
TrainNet
CLS
PRINT "Now see how the NN performs after training"
RunNet
SLEEP
END

SUB DoNet()
    DIM sum AS SINGLE
   
    'Run through nodes in HL
    FOR i = 1 TO NUM_HL
        sum = 0
        FOR j = 1 TO NUM_IL
            sum = sum + hl(i).wt(j) * curImg(j)
        NEXT j
        hl(i).act = sig(sum - hl(i).th)
    NEXT i
   
    'Run through nodes in OL
    FOR i = 1 TO NUM_OL
        sum = 0
        FOR j = 1 TO NUM_HL
            sum = sum + ol(i).wt(j) * hl(j).act
        NEXT j
        ol(i).act = sig(sum - ol(i).th)
    NEXT i
END SUB

SUB InitNet()
    RANDOMIZE TIMER
   
    'Hidden Layer
    FOR i = 1 TO NUM_HL
        FOR j = 1 TO NUM_IL
            hl(i).wt(j) = (RND - .5) * .1
        NEXT j
    NEXT i
   
    'Output Layer
    FOR i = 1 TO NUM_OL
        FOR j = 1 TO NUM_HL
            ol(i).wt(j) = (RND - .5) * .1
        NEXT j
    NEXT i
END SUB

SUB TrainNet()
    DIM numCycles AS INTEGER
    DIM current AS INTEGER
    DIM j AS INTEGER
    DIM sum AS DOUBLE
   
    INPUT "How many training cycles", numCycles
    FOR i = 1 TO numCycles
        j = 1
       
        current = INT(RND(1) * 10)
           
            FOR y = 1 TO IMG_WIDTH
                FOR x = 1 TO IMG_HEIGHT
                    curImg(j) = img(current, x, y)
                    j = j + 1
                NEXT x
            NEXT y
           
        DoNet
       
        'Calculate OL Error
        FOR j = 1 TO NUM_OL
            IF j = current + 1 THEN
                ol(j).e = (1 - ol(j).act) * GAIN * ol(j).act * (1 - ol(j).act)
            ELSE
                ol(j).e = (0 - ol(j).act) * GAIN * ol(j).act * (1 - ol(j).act)
            END IF
        NEXT j
       
        'Calculate HL Error
       
        FOR j = 1 TO NUM_HL
            sum = 0
            FOR k = 1 TO NUM_OL
                sum = sum + ol(k).e * ol(k).wt(j)
            NEXT k
            hl(j).e = GAIN * hl(j).act * (1 - hl(j).act) * sum
        NEXT j
       
        'Adjust Weights
        FOR j = 1 TO NUM_OL
            FOR k = 1 TO NUM_HL
                ol(j).wt(k) = ol(j).wt(k) + LR * ol(j).e * hl(k).act
            NEXT k
            ol(j).th = ol(j).th + LR * ol(j).e
        NEXT j
       
        FOR j = 1 TO NUM_HL
            FOR k = 1 TO NUM_IL
                hl(j).wt(k) = hl(j).wt(k) + LR * hl(j).e * curImg(k)
            NEXT k
            hl(j).th = hl(j).th + LR * hl(j).e
        NEXT j

       
        PRINT ".";
    NEXT i
END SUB

SUB RunNet()
    DIM quit AS Byte
    DIM num AS Byte
    DIM j AS INTEGER
   
    j = 1
   
    INPUT "Which number do you want to test?", num
   
    FOR y = 1 TO IMG_WIDTH
        FOR x = 1 TO IMG_HEIGHT
            curImg(j) = img(num, x, y)
            j = j + 1
        NEXT x
    NEXT y
               
    DoNet
   
    FOR i = 1 TO NUM_OL
        PRINT "Value of output node ";
        PRINT i - 1
        PRINT ol(i).act
    NEXT i
END SUB

FUNCTION sig(x AS DOUBLE) AS DOUBLE
    sig = 1/(1 + EXP(-GAIN * x))
END FUNCTION
[/syntax]I left out the DATA statements.

Interested readers may want to learn about the already closed
challenge-with-no-entries that triggered this thread:
http://forum.qbasicnews.com/viewtopic.php?t=11164

And here's a link to some neural networks and cellular automata sources
dug up from my deep & undisturbed QB-vaults, now updated for FreeBasic:
http://home.graffiti.net/vspickelen:graffiti.net/index.html

See you next time 'round,
vspickelen
Logged
Torahteen
Ancient Guru
****
Posts: 744



« Reply #24 on: January 04, 2006, 01:10:37 PM »

Thanks spick. I was so darn close. I hate those little bugs. Sorry about the challenge-with-no-entries. I was going to try to enter after I finished this. Thanks for all the help. Hmm... now what should I try to make a neural network for?  Tongue Hopefully you won't have to help me anymore.
Logged

quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
vspickelen
Member
*
Posts: 26



WWW
« Reply #25 on: January 05, 2006, 02:46:59 PM »

Quote from: "Torahteen"
I was going to try to enter after I finished this.

So just ignore the shutdown and keep on postin'!

vspickelen
Logged
Torahteen
Ancient Guru
****
Posts: 744



« Reply #26 on: January 09, 2006, 12:21:25 AM »

BTW, the reason why I was having trouble understanding the weights problem, was because I was reading Richards guide to NNs. He has 3 sets of weights, because he doesn't consider the test pattern as an actual layer. He has three layers, input, hidden, and output. He feeds the pattern into the input layer, which has its own set of weights.
Logged

quote="Deleter"]judging gameplay, you can adaquately compare quake 4 with pong[/quote]
Pages: 1 [2]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!