Danbooru

Installed Danbooru on debian, but can't upload images

Posted under General

Boy ROR is a whore.

It seems like things should work... but when I try to upload an image it gives me the failbooru message:

"Something broke. Either there's a bug or the database timed out. An email has been sent to the developers."

I don't see anything notable in the nginx, unicorn, or postgresql logs besides a warning or two.

What am I doing wrong?

Also... should I just install 2.0 ? Using 1.x trunk right now...

I had serious issues with unicorn not properly recognizing the version of rails I had installed... finally I used gem to install 2.3.5 and it started working... except for the upload image thing... image shows up in public/data but not in the db.

Edit: Oops... I do see this in the production.log:

SocketError (getaddrinfo: Name or service not known):
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh/transport/session.rb:66:in `initialize'
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh/transport/session.rb:66:in `open'
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh/transport/session.rb:66:in `initialize'
/usr/lib/ruby/1.8/timeout.rb:53:in `timeout'
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh/transport/session.rb:66:in `initialize'
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh.rb:186:in `new'
/var/lib/gems/1.8/gems/net-ssh-2.2.1/lib/net/ssh.rb:186:in `start'
/var/lib/gems/1.8/gems/net-sftp-2.0.5/lib/net/sftp.rb:31:in `start'
app/models/post_methods/file_methods.rb:22:in `distribute_file'
app/models/post_methods/file_methods.rb:20:in `each'
app/models/post_methods/file_methods.rb:20:in `distribute_file'
app/controllers/post_controller.rb:64:in `create'
passenger (3.0.9) lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'
passenger (3.0.9) lib/phusion_passenger/abstract_request_handler.rb:513:in `accept_and_process_next_request'
passenger (3.0.9) lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
passenger (3.0.9) lib/phusion_passenger/classic_rails/application_spawner.rb:321:in `start_request_handler'
passenger (3.0.9) lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `send'
passenger (3.0.9) lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `handle_spawn_application'
passenger (3.0.9) lib/phusion_passenger/utils.rb:479:in `safe_fork'
passenger (3.0.9) lib/phusion_passenger/classic_rails/application_spawner.rb:270:in `handle_spawn_application'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:357:in `__send__'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:180:in `start'
passenger (3.0.9) lib/phusion_passenger/classic_rails/application_spawner.rb:149:in `start'
passenger (3.0.9) lib/phusion_passenger/spawn_manager.rb:219:in `spawn_rails_application'
passenger (3.0.9) lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
passenger (3.0.9) lib/phusion_passenger/spawn_manager.rb:214:in `spawn_rails_application'
passenger (3.0.9) lib/phusion_passenger/abstract_server_collection.rb:82:in `synchronize'
passenger (3.0.9) lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
passenger (3.0.9) lib/phusion_passenger/spawn_manager.rb:213:in `spawn_rails_application'
passenger (3.0.9) lib/phusion_passenger/spawn_manager.rb:132:in `spawn_application'
passenger (3.0.9) lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:357:in `__send__'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
passenger (3.0.9) lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
passenger (3.0.9) helper-scripts/passenger-spawn-server:99

Updated by Bapabooiee

I think the problem is that the code is, by default, rigged to distribute uploaded images to other servers on the internal network via ssh/sftp. But obviously, you're not doing anything like that, so it just conks-out.

Danbooru's codebase makes assumptions like that in various places, where it's suitable for the main deployment on danbooru.donmai.us, but not for someone else who wants to take the code and use it. So when you deploy Danbooru, you'll need to go through a little trial-and-error until you work those kinks out.

Anyway, as far as a fix, disabling the distributing of uploaded images should do it: Open up app/models/post_methods/file_methods.rb and comment-out this line:

m.before_validation_on_create :distribute_file

Restart the Danbooru instance and see what happens.

And as far as using Danbooru 2.x, I'd recommend waiting until it's production-ready, since it's still in active development and somewhat buggy.

Thanks. That seems to have done the trick.

I stupidly was impatient and I started to reinstall all of my ROR stuff and it took me HOURS to get it working again sigh. I dunno if it is just Debian's ROR or ROR in general, but man it is a mess.

I'd say overall this was easy, EXCEPT for ROR's incredible quirkiness which can make it super extremely annoying. I'm still not sure what I did exactly to fix some of the things.

I did find the following links somewhat useful:
http://stackoverflow.com/questions/1442834/missing-the-rails-2-3-4-gem-even-though-its-installed
(The rails 2.3.5 w/ rack 1.0.1 is the only combo I can get to work)

and:
http://www.42.mach7x.com/2009/03/30/upgrading-to-232-uninitialized-constant-applicationcontroller/

I also got a lot of info looking at these three different non-functional install scripts... None of them work on their own or even together, but running them does get a lot of it going...

http://pastebin.com/mZScS2f2
http://pastebin.com/6k3rcxmr
http://www.zero0ne.de/blog/wp-content/uploads/2011/04/INSTALL.debian.txt

Maybe someone can make an install script that actually works, so people like me who are inexperienced with ROR don't lose their religion trying to get it to work.

Edit:
Oh, I've got one more request also... if anyone has already written a script for pulling images with the tags I want from other booru's and automatically uploading them to my booru keeping the metadata I'd certainly appreciate it so I don't have to write it myself.

Updated

Edited out first "problem" I figured out myself.. still would like to disable the SSD thing entirely though. Even though I found an easy enough workaround.

Edit:
For now I just made a fake ssd dir... but I'm still having the problem with the dashboard not showing any information or list of users and I can't find any errors being generated in the logs.

Edit2: Nevermind... it appears I just needed to actually do something with the other user I created. I was assuming I'd just be able to list all users from the dashboard.

Updated

Yeah, expecting an SSD/SSD path is another one of those "assumptions" I was talking about.

I'd help you out with your issues, but I haven't deployed Danbooru 1.x in a while, so I can't see what the causes of your problems are.

A couple suggestions:

Try checking-out an earlier revision of the code and using that. Lately, Danbooru 1.x's code has been in "maintenance-only" mode because of Danbooru 2's development, so it's only been getting ad-hoc fixes and backported features. I'd recommend checking-out the 1.18.0 or 1.17.0 branches if you want to go this route.

Or, if you're not very particular, you can use Gelbooru instead, a PHP-based Danbooru clone that uses MySQL as its database. From what I can recall, deploying it was mind-numbingly easy, and largely problem-free.

Good luck.

It's okay. If I had known the gelbooru source was available I would have tried it first honestly.

And the problem with the dashboard was just me not knowing how it worked. ;)

Making a new VM to test gelbooru on now.

Thank you very much for your help... now if I could just find someone who has a script that will leech danbooru and gelbooru for their tags and fill in my database automatically with them. ;)

If it's of any use, you can get a database dump of Danbooru from forum #40051. Though you should be wary that it's missing some data (like users' personal messages), and is dated at over a year old.

Nonetheless, you could write a script to import the data you want from it, and see if you can make any use of it.

Obviously, it doesn't include the images themselves, but since you'll have their MD5 data, you'll be able to locate and download them* from Danbooru (and Gelbooru too, possibly, since that site imports posts from Danbooru) as the image filenames are their MD5 sums.

*If you decide to go with this, be nice, and don't hammer the servers too hard.

1