What’s wrong with CNN regression?

In my koi17 project, I am using convolutional neural network regression model.

My model is working with standard VGG16 architecture.

But the performance of NN is not enough for my goal, and it also seems under fitted too.

So I decided stack more layer and change the architecture to ResNet. I stacked 4 of the residual block.

In a usual classification problem, ResNet must be working much better than VGG… But in my case, it wasn’t.

ResNet worked worse then VGG, and then I tested shallow DenseNet too, but it showed up higher bias and test-acc is overfitted too…

I think deeper CNN is not a good solution in regression problem because ConvNet ignores geometric placements.

 

I have no solution to solve this problem yet…

 

-Update 2017-11-16

I found what is the problem with my model. I made too narrow space for CNN. I passed 4×4 image for last of 5~6 layers and it made overfitting because filter size was 3×3, so 9 of 16 nodes were fully connected. So it increased more fully connected ratio in my network so it got overfitted.

I removed some pooling layer on top of the model so the last layer got the 8×8 image to compute. And it solved the problem.

Heejun Lee

Hi Everyone! I think we need more active people in CODEX! Everybody, enjoy my codex, and comment!

2 thoughts on “What’s wrong with CNN regression?

  1. Wow! I gather that you’ve run into a problem and tried to solve it using various related methods but none of them worked. Other than that, though, I didn’t actually understand anything in those sentences! LOL

    1. Haha, it is always happening in programming. Get some problem, and use some various and advanced solution to solve it, but all of them is not working. My experience is way too little to solve problems by just one try. Other than that, I think my posts are not a friendly description too. I should explain what I am saying more next time. (*p ω q*)

Leave a Reply

Your email address will not be published. Required fields are marked *