Syncing Interworx With External DNS Servers [HOWTO]

[gethosted]

Installing External DNS For Your Interworx Server(s)
If you are like us and have several Interworx servers that house clients, and
want those clients to be able to change their DNS settings through the Interworx cp you are pretty much out of luck without some serious hacking.

Well, we had the same problem and decided to solve it. Here is a short HOWTO on syncing all of your Interworx servers with an external tinydns server(s).

Here is what is needed:

A server, or servers with the Interworx CP installed. Steps will be color coded green
A server, or server with an Interworx compatible OS installed. Steps will be color coded blue
About an hour
A brain with a mass of at-least 525 grams. For more information on calculating brain mass, please visit: http://www.roadtoreading.org/personal/brain_power.html
Some sense of humor


For everyone's clarification here are the terms that we will be using in this HOWTO:

CPServer = Refers to one of your Interworx servers that you would like to sync with a "DNS Server"(see next item).
DNS Server = Refers to one of your external DNS servers that will actually be publishing the data from your CPServer(s).
Tinydns = The actual DNS program that runs on your "DNS Servers" (see item above).
*word* = The inclusion of "*" marks around a word means that the word is emphasized.
// = When a line starts with these symbols: "//" it is a comment...Don't type this on the command line...It is just me trying to help you understand a specifc command.
patience = The capacity, quality, or fact of being patient
hosed = "(Slang.) To attack and kill (a computer entity), typically by use of a poorly executed user input, example: Peter hosed the computer by typing: (rm -rf /)." THIS IS ALWAYS BAD


Warning: This HOWTO and related scripts are provided with absolutely positively no warranty. Any damage they do to your system we are indeed sorry for, however we cannot be help liable. Proceed at your own risk....(it really isn't that bad).

Ok, throw on your brain protectors, grab a Mountain Dew, and perhaps say a small prayer...lets begin.

Step #1 - Install tinydns(and related apps) on your DNS Server(s)
Since this HOWTO is relatively platform inspecific (you can make it work on a FreeBSD box), we will use the standard install(compile).

Hose Potential Warning: These steps must NOT be executed on your CPServers!

Visit: http://cr.yp.to/djbdns/run-server.html for the exact step by step instructions on installing tinydns.

Step #2 - Install PHP On Your DNS Server(s)
Yeah yeah, I know...a simple SH would have been easier...but I like PHP, support it's use in shell scripting....and Interworx uses it as well. It was just a natural selection.

The script for syncing the CPServer(s) to the DNS Server(s) does not need any special modules installed. We just need to do a standard install.


mkdir /var/src
mkdir /var/src/tar
cd /var/src/tar

wget http://techdev.gethosted.net/php/php-current.tar.gz
cd /var/src
tar zxf tar/php-current.tar.gz

cd php*
./configure
make
make install


You should see quite a few lines run by, and hopefully no errors...If there were errors take a look here: http://www.php.net/manual/en/install.php


Step #3 - Grant Access To Your DNS Server(s) On Your CPServer(s)
Interworx comes with a pretty nifty XML access system that lets other computers access the DNS records it currently publishes via a PHP file. This option is *disabled* by default, however we can selectively add in the authorized IPs by editing a simple text file. Lets begin:


// This assumes you are on your CPServer and at a command line
cd /home/interworx/etc/
vi remote-access.ini

Great, you should now have a file that looks a little like this:


[dns]
exportdns.allow=""


We want it to look like:


[dns]
exportdns.allow="xxx.xxx.xxx.xxx"


Please replace the "xxx.xxx.xxx.xxx" with the IP of your DNS Server(s). If you have multiple DNS Servers that will be accessing this CP Server seperate the IP addresses with the "|" symbol.

Go ahead and save that file....We are all done with the CP Server


Step #4 - Download The DNS_SYNC Script On Each DNS Server
Ok, we don't have much left....Now we just need to download the DNS_SYNC script and configure it.


mkdir /usr/local/dns_sync
cd /usr/local/dns_sync
wget http://techdev.gethosted.net/dns_sync/dns_sync
chmod 770 dns_sync


Ok, great...if everything went ok we just downloaded DNS_SYNC and made it executable.

Now we need to configure it for your specific installation:


vi dns_sync


You should now see the actual syncing script, and there should be quite a few comments. Read each of those comments, the defaults usually are *ok* for most installations, but if you have errors while syncing make sure they are correct.

After you have read through the comments, and made the corresponding changes save the file and return back to the prompt.

IMPORTANT: The last variable in the variable section is a variable called: $TEST_MODE . This variable defaults to: true (Test Mode is one). Test mode will just download the DNS data from your CP servers, however it will not overwrite your default tinydns datafile, nor will it create a .cdb file.

Once this variable is set to: false , the tinydns datafile will be overwritten with the dns_sync created one and dns_sync will create a new cdb file.

Ok, now that we have the configuration stuff taken care of we need to test the script to see if everything is running. Go ahead and type the following from a command prompt:


/usr/local/dns_sync/dns_sync


You *should* see some status messages outputted to the shell screen while dns_sync is working.

Here are some crucial lines:


##### XML Data File Successfully Downloaded For: yourserver.domain.com #####


That one is very good.....Jump up and down 10 times if that one is displayed!


####### - {insert number here} records added for server: yourserver.domain.com


That is also good...10 more jumps for you!


##### Writing yourserver.domain.com data to new tinydns datafile.....Done


That is the best one...That means all of the tinydns data has been successfully parsed and added to a temporary datafile.

You can feel free to take a look at that datafile if you would like to make sure all of the configuration is correct. You can take a look at it by doing:


cd /usr/local/dns_sync/data
cat data


Once you are happy that everything is correct, go ahead and set the $TEST_MODE variable in the dns_sync script to false, and run the command again:


/usr/local/dns_sync/dns_sync


There should be two additional lines outputted by the script...they are:


##### Copying new data to tinydns production file
##### Running the tinydns makefile.......Done
##### dns_sync completed successfully


Now everything should be sync'd correctly!

We have the script sync with all of our Interworx server(s) every 5 minutes. The bandwidth transferred is really negligible, and the ability for users to create/edit subdomains from within Interworx is a huge plus!

If you received any errors during the process feel free to post them to this thread or to e-mail me: techdev@gethosted.net

This is a beta release, so feedback/bug reports would be much appreciated! Enjoy!

Peter

[IWorx-Chris]

This is fantastic gethosted and not many people know of the xml-export ability so at least we have a solid 'howto' now so 1) people know it exists and 2) it's not mega mysterious anymore :).

Thanks a ton,

Chris

[roman]

i'm not gonna use it for the moment, but that's a great tutorial !!!!
thank you very much, you make some great work to help this comunity :) :)

[pascal]

you've done a great job. Thta's cool


If you are like us and have several Interworx servers that house clients, and
want those clients to be able to change their DNS settings through the Interworx cp you are pretty much out of luck without some serious hacking.


haha, maybe I've jumped to high when I was young, because I do not understand this sentence


want those clients to be able to change their DNS settings through the Interworx cp


yes. I want, but it is already the case ?
to have an external dns server will not give the ability to those clients to change their dns without an other script giving them an interface to do this ?

I understand what you are doing, but I do not undersatand this sentence..... ouin ouin :confused: :(

somebody could explain me :rolleyes:

Thanks
Pascal

[gethosted]

Pascal,
Basically this script allows you to sync all of the Interworx DNS information with an external DNS server.

This is often referred to in the Bind world as a master-slave sync...or a zone transfer.

The best way to describe it is to ask a question:

Suppose you have 50 Interworx servers that house client websites, etc. You want each account on those 50 servers to be able to add subdomains, etc.

Now, the not so great way of doing this would be to register 100 NS servers with your registrar(i.e. ns1-ns100). Then, each Interworx server would have its own ns server associated to it(i.e. ns20 ns21 for one server). Your clients would have to put ns20 and ns21 into their nameserver settings at their registrar. That would be the *wrong* way to do things.

The right way, and the way I would suggest doing it would be to have two "external" DNS servers for those 50 interworx servers. These servers would do nothing besides serve DNS information.

In order to "sync" all of those 50 Interworx servers to the external DNS servers you would need to use the script I wrote above.

This method would allow you to tell your clients to just use: ns1 and ns2.yourdomain.com as their nameservers.

Does this make sense?

Peter

[pascal]

lol, ok now I understand.

"to be able to change their DNS settings through the Interworx cp " disturb me a lot.

I thought you talking about a more functionnal tool to allow the users to edit their dns information (as nodeworks does for esample)

You absolutly right.

And I'll tell more, with this script ypu may now easliy create a primay dns server on a dedicated dns box1 and the secondary on an other dedicated dns box, that is also great.

I was thinking doing this only by rsync the /var/djbdns/tinydns/root/data.cdb, but there was some pbms doing this.

So your script is really welcome

Great job

Pascal

[pascal]

Now It's time for me to play with this !!!

I've decided to have only 2 dns servers regitered and not 1 hundred :)

Ok a question : Is it possible that my DNS server are hosted on a Interworx-cp server ? I do not have so much money to have a dedicated box only for DNS.

So I'd like to share one box as dns server and interworx-cp

Is it ok ?

Thanks

Pascal

[pascal]

Could an Interworx member confirm me that I may use the /var/djbdns/tinydns/root/data rather /etc/tinydns/root/data to add the data that come from the interworx xml dns-export ?

I'd like to use a box with Interworx as my main dns server and includ in it the dns informations that come from others interworx cp server

Is it ok ?

[IWorx-Chris]

Could an Interworx member confirm me that I may use the /var/djbdns/tinydns/root/data rather /etc/tinydns/root/data to add the data that come from the interworx xml dns-export ?


I'm a little confused by the question Pascal. /etc/tinydns/root/data?

Chris

[pascal]

In the gethosted script "dns_sync" it adds dns data somewhere in /etc/tinydns/root/data

Also he told us

Step #1 - Install tinydns(and related apps) on your DNS Server(s)
Since this HOWTO is relatively platform inspecific (you can make it work on a FreeBSD box), we will use the standard install(compile).

Hose Potential Warning: These steps must NOT be executed on your CPServers!

Visit: http://cr.yp.to/djbdns/run-server.html for the exact step by step instructions on installing tinydns


So if I understand well it means that he installed a new dns server from scratch. Maybe by default the data is are store in /etc/tinydns/root/data ?


Hose Potential Warning: These steps must NOT be executed on your CPServers!


Not sure why, my idea is that it is possible to have my main dns server on an interworx box as this one has already tinydns etc ....

Also not sure to have to do these steps http://cr.yp.to/djbdns/run-server.html

I could change the directory set in the gethosted script as he has commented all the codes. But before doing this I'd like to be sure I can export all the dns data from my others interworx-cp boxes to one boxe with intetrworx-cp ?

If yes, is there some things I have to change, to be care with ?

Thanks

[gethosted]

Pascal,
Personally, I would run two instances of tinydns on that second Interworx box. One would be the one that Interworx comes installed with, and the second I would install myself. It is very easy to do, just assign your machine another IP address and create a new directory for the tinydns installation.

Once the install has been completed simply edit my script to point to that new tinydns installation and run as usual. It should query all of the information from your servers and add it to that new tinydns install.

Hope this helps,

Peter

[pascal]

It is very easy to do, just assign your machine another IP address


Not sure to understand. When I look the default interworx's tinydns install it seems tinydns is on ("listening") all IP.

What do you mean by assign your machine another IP address ?

Ok say that I have two boxes.
Box1 is today my dns server box + interworx-cp
Box2 is a new one (also with interworx-cp)

I install a new tinydns instance on the box2. Right ?
Then I read the export-dns.php from box1. Right ?
But I also have to read the export-dns.php file that is on the same box, so box2 ? inst it ?

After this, I'll have to register in my registrar this new dns server. And change all existing account to this new dns server. Every account won't be available for 12h/-48h00 the time all world dns server been updated. Right ? It's also why I would prefer keep my existing dns server.

Why do you think it should be best to create a second tinydns server on the same box ? to not interfer with the interworx one ?

Thanks a ton for your help

I'm really not so good in dns management

Pascal

[pascal]

Chris ? Paul ?

What is your comment. What Interworx would do or thing is the best to do ?

I'm waiting for your opinion before starting to use this box.

Also, today I have registered 2 names server (primary and secondary) at my registrar, would it be possible to uses these two IP for the new (second) tinydns instance (in case I can't use the existing one) ?

What is the difference between this http://cr.yp.to/djbdns/run-server.html and the interworx-cp installation ?

If I follow this http://cr.yp.to/djbdns/install.html and this http://cr.yp.to/djbdns/run-server.html would be ok and suffisant ? Any pbm to do this with my on my two registered dns IP ? no conflict with interworx ?

If not thanks for your assistance and to provide a mini howto if you can.

I'd like to understand. Apparently on an interworx box there is a dns server on every IP ?
Indeed I have a data.cdb file in every /service/tinydns-xx.xx.xx.xx directory (where xx.xx.xx.xx is all the IP adress hosted on my box)

Thanks a ton for your help

Pascal

[IWorx-Chris]

Pascal,

There's so many questions here I'll just lay out how iworx-cp works and how to sync 2 servers easily.

iworx-cp runs an instance of tinydns (tinydns is the iterative dns server) on every IP bound to your server by default.

If you nslookup using *any* IP on the server as a dns server you'll get a response for sites hosted on the server.

You can use any of these IPs as a dns server, so any is valid to provide (along with their DNS'd name) to a domain registrar.

gethosted's script is a good way to sync servers using iworx-cp's DNS database. If you simply want a 2nd DNS server which mirrors your iworx-cp box and *only* the iworx-cp box all you really have to do is the following:

- setup tinydns on the 2nd (non-iworx) server (using any docs you find, the cr.yp.to docs are fine here and you can use them).

- every so often just copy /var/djbdns/tinydns/root/data.cdb from the iworx-cp server to your new tinydns server's tinydns/root directory. Nothing else is needed, no restating, nothing.

That's the simplest method to get 2 dns servers with the same data answering iterative queries.

Chris

[pascal]

Thanks Chris

The only thing I'd like is to use the DNS server I already have on my Interworx-cp as my main dns server for ALL other boxes. I mean I have one boxe with interworx-cp and I have registered at my registrar 2 IPs of this one to be my main dns server.
Now I'd like that all others interworx-cp boxe update the data.cab of the first dns server owned by the first box. In fact exactly as gethosted howto does, but only keep my exisitng dns server as the main one, not creating a new one on an other box

I hope what I wrote is understanding ?

So the only real question I have is to know if I can use my existing dns server (the ones on my first interworx-cp) as the main one ?

Thanks a ton

Pascal

[IWorx-Chris]

"main" or "primary" isn't really distinguished in djbdns. If you copy the data.cdb from the iworx-cp box to all others then you could call the iworx-cp box the "main" one since it holds all the base data. So the short answer is, yes, this is possible.

Chris

[pascal]

ok

but you mean if I copy the data.cdb from, and not to, all others box to the main one, isn't it ?

[IWorx-Chris]

copy data.cdb from whatever box you want to deem "main" to all boxes that you want to deem "secondary".

Chris

[pascal]

Ok sorry Chris, but I understand that maybe you don't understand what I mean. It's sure it is because I have some pbms to explain. Sorry for that. Really.

When I tell MAIN I don't mean primary. My primary dns server and my secondary (in fact an other primary if I understand well) dns server is defined on 2 IPs on box 1.
I'd like to use these 2 dns server as my dns servers for ALL interworx-cp boxes. So every changes on other boxes as to be copied (replicated) to these two dns server.

Ok I understand now that /var/djbdns/tinydns-xxx.xxx.xxx.xxx/root/data.cdb is a link to /var/djbdns/tinydns/root/data.cdb (I didn't figure out this)

So, I'll do some tests.
Allow on my box1 the access from box2

[dns]
exportdns.allow="xxx.xxx.xxx.xxx"


modify the gethosted script to get data from box2 from /var/djbdns/tinydns/root/data.cdb to modify /var/djbdns/tinydns/root/data.cdb on box1

Anyway

Thanks a ton for your patience and your time past with me.

Pascal

[pascal]

Ok finally it seems to work

I have two Iworx-cp box, my box1 is my dns server

I have edited on box2
[dns]
exportdns.allow="xxx.xxx.xxx.xxx" (= ip of box1)

Changed in the gesthosted script :
-- the tinydns path to /var/djbdns/tinydns
-- the iworx server to my box2 ip


And everything works fine

I'm just afraid now to forget to also add my box1 as an iworx server in the gethosted script. Indeed, I afraid to have updated my data.cdb file with only the dns data from box2 and so lost and erased the data from box1. In fact the script look in data to keep existing data, but as in data there is nothing the first time, I assume than Interworx create directly data.cdb without adding not compiled (clear information) information in data

I think i'd also add the box1 as iworx server to sync from

Lol I've asked a ton of questions before trying, I had to test by myself. Unless there is this last questoin to resolve


Pascal

[pascal]

For gethosted

To make the file data in data.cdb you use Make. Why not use the tinydns-data pgm ?


// compile the changes with the "make" command
echo "##### Running the tinydns makefile.......";
exec("cd $TINYDNS_DIR/root/; $MAKE_PATH");
echo "Done\n";
echo "##### dns_sync completed successfully\n";


Also, I'm not sur this will create the data.cdb file ?
by default, $MAKE_PATH = '/usr/bin/make'

Maybe it should the MakeFile in the root tinydns directory (that call tinydns-data) ?

Thanks
Pascal

[pascal]

I found that all dns information are stored in /home/interworx/var/lib/dns/data

So in the gethosted script I let it get the file from my other box and I add the information from my current box simply by doing a

cat /home/interworx/var/lib/dns/data >> /var/djbdns/tinydns/root/data


Like this the final data file contains all dns data from ALL others box + the dns data of the current box.

I think that after I'd simply run tinydns-data to create the data.cdb !?!

I have to ask to an interworx member more about /home/interworx/var/lib/dns/data
like when does it updated ? is it always up to date ?

And what difference it may have between /home/interworx/var/lib/dns/data.cdb and /var/djbdns/tinydns/root/data.cdb

Oufff... I begin to see the end

But maybe what I did is not necessary if the make process work without tinydns-data but make and if it ADD data in data.cdb rather than replace data in data.cdb

Pascal

[pascal]

which data.cdb I have to update ?
/var/djbnddns/tinydns/root/data.cdb or
/home/interworx/var/lib/dns/data.cdb

Indeed, I create the /var/djbnddns/tinydns/root/data with the dns data from my box2 and paste in it the the data from box1 (/home/interworx/var/lib/dns/data)
Then I create /var/djbnddns/tinydns/root/data.cdb by running tinydns-data.
During few minutes the /var/djbnddns/tinydns/root/data.cdb has a size equal at 51130 then few minutes later it has a size less important equal at 50510.

My opinion is that interworx update automaticly the /home/interworx/var/lib/dns/data.cdb and use it? ins't it ?

So what are the differences between the twice data.cdb ?
Which one I have to update ?

In this case I may understand that it isn't possible to update the data.cdb with others informations that exists in nodeworx. It's surely why it would be better to run an other instance of tinydns.

Thanks

Pascal

[IWorx-Chris]

InterWorx-CP dumps the data in its internal DB to /home/interworx/var/lib/dns/data.cdb and then this file is copied to /var/djbdns/tinydns/root which makes it 'live'.

Chris

[pascal]

Ok it's why for All my updates in /var/djbdns/tinydns was erased.

I have created a new instance of tinydns on my two IP
in

/etc/tinydns
/etc/tinydns-65.110.36.142
/etc/tinydns-65.110.36.143

with tinydns-conf

Created a symlink in /service to these directory
tinydns2
tinydns2-65.110.36.142
tinydns2-65.110.36.143

and started them.

Before I have deleted the 2 exisitng symlink in /service that pointed on /var/djbdns/tinydns

I also did as interwox do, created a symlink in /etc/tinydns-65.110.36.142/root to /etc/tinydns/root same for 65.110.36.143

I finally changed the gethosted script to get data from others boxes and get data from the current box from /home/interworx/var/lib/dns/data

So I have in /etc/tinydns/root/data all the data from box2 and box1

Apparently it is ok.

Now I just have to wait to see if my resitered dns server are always serving DNS data (in fact if the domain name on box2 become reachable it will mean it works)

Thanks you Chris and GetHosted

Juts hope I dind't broke my current tinydns install

Is that sound OK ?
Is there somethink to do with the DNSCACHE ?

Pascal

[pascal]

Finally here is some last infos about the interworx tinydns install and how I did the setup of a new one

Interworx create a dnscache for 127.0.0.1

Interworx create a default /var/djbdns/tinydns with a default IP 1.2.3.4

tinydns-conf tinydns dnslog /var/djbdns/tinydns 1.2.3.4

they do not create a service for this one. So it means they do not create a symlink in /service/

They remove in /var/djbdns/tinydns all directories unless the root one

The data.cdb in /var/djbdns/tinydns is the exact copy of /home/interworx/var/lib/dns/data.cdb

They use this data.cdb as the main one. Every new server created on every IP you have will point on this data.cdb

So they create a dns serveur for every IP you have

tinydns-conf tinydns dnslog /var/djbdns/tinydns-xxx.xxx.xxx.xxx
where xxx is the IP


in all these new folders they create a symlink to /var/djbdns/tinydns/root
It means that there is only ONE data.cdb on for every IP.

Then they create the services by creating a symlink in /service/ for all these new server.

ln -s /var/djbdns/tinydns-xxx.xxx.xxx.xxx /service/tinydns-xxx.xxx.xxx.xxx



So here is what I fund after a lot of asks and a lot of research
Considering this here is how I have setup a new dns server on my box

1- I have deleted all symlink in /service/
I have no more /service/tinydns-xxx.xxx.xxx.xxx

2- I have stopped the service and deleted all /var/djbdns/tinydns-xxx.xxx.xxx.xxx

in every /var/djbdns/tinydns-xxx.xxx.xxx.xxx
svc -dx . log

then

rm -rf /var/djbdns/tinydns-xxx.xxx.xxx.xxx
[/quote]


3- I have keep the /var/djbdns/tinydns as it is in fact the one who hold the dns data


4- I have created my new tinydns server

tinydns-conf tinydns dnslog /etc/tinydns2-xxx.xxx.xxx.xx1 (where xxx is my regitered primary dns IP)

it means that I have an independant root/data and root/data.cdb in this folder

same for my secondary dns server

tinydns-conf tinydns dnslog /etc/tinydns2-xxx.xxx.xxx.xx2 (where xxx is my regitered primary dns IP)

but I have deleted the root folder in /etc/tinydns2-xxx.xxx.xxx.xx2 and created a symlink root that point to /etc/tinydns2-xxx.xxx.xxx.xx1

So I share with my primary server and my secodnary server the same data and data.cdb

then I have created the service by creating the a symlink in /service/

ln -s /etc/tinydns-xxx.xxx.xxx.xx1 /service/tinydns2-xxx.xxx.xxx.xx1
ln -s /etc/tinydns-xxx.xxx.xxx.xx2 /service/tinydns2-xxx.xxx.xxx.xx2


and started them

Well, now I think everything is fine.

In the gethosted script I have change the tinydns path to /etc/tinydns-xxx.xxx.xxx.xx1

The idea was to copy the entries in /home/interworx/var/lib/dns/data to /etc/tinydns-xxx.xxx.xxx.xx1 after the script has gotten the data from others boxes

So it means that in /etc/tinydns-xxx.xxx.xxx.xx1/root/data I'll have :
- All dns data get from the dns-export.php file from all others boxes
- Then the dns data of my current interworx box

To do this I've only added these lines


// Pascal MAJ ================================
exec("echo '####### start padawan.carat-hosting.com data #######' >> $TINYDNS_DIR/root/data");

exec("cat /home/interworx/var/lib/dns/data.save >> $TINYDNS_DIR/root/data");

exec("cp $TINYDNS_DIR/root/data.cdb $TINYDNS_DIR/root/data.cdb.save");
// Fin Pascal ==================================


In conclusion, I now understand how interworx has installed and run the tinydns server.

Thanks to Gethosted for this great script and thanks to interworx team for having provide me some useful informations

Hope it may help some of you

Pascal

[pascal]

The last problem is that the XML file generated by interworx doesn't include ALL informations of the SOA records.

So waiting for this, I no more use the gethosted script and the interworx xml data.

I have created public key to access my others server from my main one and I use sftp to retrieve from ALL others server the /home/interworx/var/lib/dns/data file

Then I merge all data file received with the one on my main box and copy it in my dns server directory, then do "make" to create the data.cdb

I'd prefere use the xml file as the data are sent securely, but by the way, I'll do this method until interworx provides ALL informations of the SOA record in there xml file

Pascal

[mpcomputers]

This was VERY helpful!

Chris, will you be updating the XML output to supply the SOA record also? My secondary DNS server will not answer authoratively without it.

Thank you!

[pascal]

Glad to see that has could help you. Lol when I read again all my posts, I found them just a little confiouses :-p

About the SOA records, I also have this pbm but only for .FR zone

Pascal

[IWorx-Chris]

If it's not there we'll definitely add it. I think what we do now is auto-add it on every box we export from with default values.

I'll check it out and open a bug report if needed.

Chris

[timryberg]

This is EXCELLENT and would solve many issues I have been seeing over the horizon moving from a single box to multiple. Not quite ready to try it yet but it's nice to have when the time comes.

[pascal]

Chris or Paul

in v2.0 does the tinydns service are still recreated automaticly on ALL ips during daily Cron job ?

I ask this as I did a ssh program to remove the services on the two IP I use as MAIN/GLOBAL box dns servers (because the data.cdb of the current erase the data of others box)

Pascal

[IWorx-Chris]

in v2.0 does the tinydns service are still recreated automaticly on ALL ips during daily Cron job ?


No, since we provided a "real" interface for administering the dns servers and what they are doing we took out this auto-install feature. BUT When you add a new IP address DNS services are auto-added (to be nice :)). You can remove / stop / start any DNS service on a PER-IP basis now. So it's much more granular and hopefully admin-friendly.

Chris

[pascal]

cool, so i can stop my cronjob stoping and removing the dns services on these two ip

Great

Pascal

[xlogicgroup]

Chris,

Any updates on when SOA records will be exported? :)

[gethosted]

Hey Everyone,
Sorry to be absent for so long, but I am glad to see that people are actually using the script :-).

I remember attempting to add SOA support in the first version of this script, however I just remembered why I couldn't get it working correctly.

When the script calls the dns-export.php file, the XML that is returned for SOA records does not have all of the information being published by tinydns on the Interworx server. It appears that the Interworx server "knows" some default values for the DNS server that it adds in to make the tinydns data file, but ommits when it does a dns xml export.

For example, if I run the export script I receive the following XML output for an SOA record:


<record dir="forward" zone="gethosted.net">
<type>SOA</type>
<domain>gethosted.net</domain>
<host>gethosted.net</host>
<target></target>
<ttl>43200</ttl>
</record>


What is actually being published in the Interworx tinydns file is:


Zgethosted.net:ns1.gethosted.net:admin.gethosted.n et::7200:2048:1048576:21600


If we could get Interworx to output *all* of the data that is being published in the Interworx data files we would be in business.

Feel free to e-mail me directly if you have any specific questions.

Thanks!

Peter

:D

[pascal]

You absolutly right.

I've sent an email to chris some weeks ago with the content of the xml file. And It's why for we patiently waiting for this :)

I'm sure chris, paul and the others are working on it, and they'll give us these soa data for he next release

And a big thank you gethosted for your script. Even if I made my own, yours really help me at the beginning to understand how it may works

Personnaly I don't use the xml-export but simply scp the data.cdb from my boxes to the main one

Pascal

[xlogicgroup]

can anyone confirm if the soa records will be sent out in the next update? :)

[IWorx-Chris]

Thanks for the nudge. I'll get it coded tonight and see if we can get it in the next one guys.

Chris

[IWorx-Chris]

This is all set guys, and I've attached the new file in case you'd like to test it out now. PLEASE MAKE BACKUPS OF THE OLD BEFORE YOU TEST.

Chris

[xlogicgroup]

Thanks chris, I'll give it a whirl today..

[xlogicgroup]

Chris,

Getting an error using that file..

Parse error: parse error, unexpected T_STRING in /chroot/home/interworx/nodeworx/html/dns-export.php on line 5

[IWorx-Chris]

Can you open a ticket and provide your login info, it works ok for me.

Chris

[xlogicgroup]

Done. Thanks chris.

[IWorx-Chris]

xlogicgroup,

You mentioned that serials were missing but I do have them being exported on the SOA. Can you shoot me your XML file that has them missing when you get a sec?

Chris

[xlogicgroup]

Thanks for fixing thatup chris. Looking forward to the next update.

[xlogicgroup]

Chris,

All is working great except dns-export needs to export <target> in the SOA record. For example ns1.domain.com etc..

Any chance of getting this added in? I can hardcode it for now but having it dynamic via interworx dns-export would be great.

Thanks

[IWorx-Chris]

I'll get it added.. not sure why I didn't notice it before :S.

Chris

[xlogicgroup]

Great thanks alot..

[warp3]

With our current install (3 days old) of Iworx we are syncing 2 external (tinydns) servers by copying the data.cdb file as suggested. Ok, worx for now.
However, not really pleased with the way it works.

Our normal setup is as followed (and I hope this is also possible with tinydns/iworx):


We have 3 main webservers, all running bind (the newly created iworx box is not yet running production). Server names: NOMAD, KLINGON and VULCAN
Next to that, 2 seperate boxes that function as DNS server, also running bind, server names: NS & NS2
Alle 3 webservers fill their own zone files, setting themselves as MASTER and setting NS & NS2 as the DNS servers.

Every 4 hours all 3 webservers generate a .txt file and fill them with the zones they are master for (just the zone names, aka domainnames).
Both NS and NS2 will then place that .txt into the bind config and setting the right webserver as master, and themselves as slave.

This way (example), when a new domain is created on webserver NOMAD, within 4 hours both DNS servers have this domain in their config file. Since the actual DNS server (NS & NS2) doesn't know the zone records yet, it requests a zone transfer from NOMAD.

When a client changes the domain config through Plesk (adding / changing records) the webserver (or bind on that server) will instantly sent a notification to its slaves, beeing NS and NS2. And when NS or NS2 reboots it will do a zone transfer again from all webservers.

At this moment, we need to run 2 extra DNS servers (NS3 en NS4) to get the data from the Iworx server.

We would like to keep running the same setup (using NS and NS2). Thus having bind to get its master information from tinydns.

So basicly we just need:
- tinydns/iworx to produce a .txt file with zone names
- tinydns needs to allow a zone transfer request from NS and NS2
- tinydns to send dns notifications to its slaves (beeing NS and NS2)

Possible ??

[IWorx-Chris]

So basicly we just need:
- tinydns/iworx to produce a .txt file with zone names
- tinydns needs to allow a zone transfer request from NS and NS2
- tinydns to send dns notifications to its slaves (beeing NS and NS2)

Possible ??


The first 2 are possible, the last isn't but with #2 in place #3 shouldn't be as critical.

#1 can be done by accessing the iworx db directly or by using the xml export URL to build the text file. #2 can be done by setting up axfrdns (from the tinydns site).

Chris

[IWorx-Chris]

Thanks for fixing thatup chris. Looking forward to the next update.

This is all set in the next point release.

Chris

[blahrus]

$IWORX_SERVERS = array('cpserver.yourdomain.com'); // to add additional servers

Can I just put the ip's of the servers in there?

Thanks,
Clint


UPDATE

[root@srv001 dns_sync]# /usr/local/dns_sync/dns_sync
Content-type: text/html
X-Powered-By: PHP/4.3.11


##### Downloading XML Data File From: 216.86.157.23 #####
##### WARNING: Data file could not be downloaded for: 216.86.157.23
##### Creating Main TINYDNS Datafile
ERROR: 216.86.157.23 temporary tinydns datafile not readable

I am that far . . .

[blahrus]

Has anyone been able to make this script work from, interworx box to interworx box?

[root@srv002 ~]# /usr/local/dns_sync/dns_sync
--13:17:55-- https://216.86.157.25:2443/nodeworx/dns-export.php
=> `/usr/local/dns_sync/xml/216.86.157.25'
Connecting to 216.86.157.25:2443... connected.
WARNING: Certificate verification error for 216.86.157.25: self signed certificate
WARNING: certificate common name `interworx.info' doesn't match requested host name `216.86.157.25'.
HTTP request sent, awaiting response... 200 OK
Length: 2,154 (2.1K) [text/xml]
100%[================================================== ==================>] 2,154 --.--K/s
13:17:55 (8.38 MB/s) - `/usr/local/dns_sync/xml/216.86.157.25' saved [2154/2154]
PHP Notice: Undefined index: record in /usr/local/dns_sync/dns_sync on line 317
PHP Notice: Undefined index: value in /usr/local/dns_sync/dns_sync on line 280
Content-type: text/html
X-Powered-By: PHP/4.3.11

##### Downloading XML Data File From: 216.86.157.25 #####
##### XML Data File Successfully Downloaded For: 216.86.157.25 #####
####### - Starting to parse the xml file
####### - 8 records added for server: 216.86.157.25
##### Creating Main TINYDNS Datafile
##### Writing 216.86.157.25 data to new tinydns datafile.....Done
##### Copying new data to tinydns production file
##### Running the tinydns makefile.......Done
##### dns_sync completed successfully


Do I need to be worried about:

PHP Notice: Undefined index: record in /usr/local/dns_sync/dns_sync on line 317
PHP Notice: Undefined index: value in /usr/local/dns_sync/dns_sync on line 280



/var/djbdns/tinydns/root/data

Has the dns info from the server I am am trying to sync it's record, but the records never seem to get merged.

Any help would be great.


Thanks,
Clint

[pascal]

Hi

Just to be updated as for now I just copy the iworx cdb file from servers to main one, is the SOA is exported yet ?

Pascal

[blahrus]

I believe Chris has already fixed this.

[IWorx-Chris]

Just to be updated as for now I just copy the iworx cdb file from servers to main one, is the SOA is exported yet ?


Yep, all fields are exported.

Chris

[JayBaen]

Does the dns_sync script still exist? gethosted's path seems to have gone the way of the dodo.

Would someone be kind enough to paste the contents/attach the script?

JB

[blahrus]

ehhh yea, the script is still there . . . the guy isn't around much, we were even willing to pay him for help ($75 an hour) and he got on the server for about 10 min then left, I think the size of our project was to large . . . not sure, but I know Chris and Staff are working hard to have it built into the panel, should be out end of this year, start of next.


Thanks,
Clint

[JayBaen]

http://techdev.gethosted.net/dns_sync/dns_sync

... can't get to from here.

techdev.gethosted.net = nonexsitent domain.

Is is located somewhere else (path change, domain change) that I'm overlooking?

JB

[blahrus]

Yea, I assume he is done ;)


I have a copy at home, I will upload a copy if you like, but I was unable to get it to sync from an interworx box to an interworx box.


Thanks,
Clint

[JayBaen]

That would be great.

I'm going from Iworx box to non-Iworx (if all goes well -- or at all .. ;) )

Thanks again.

JB

[JayBaen]

<bump>

If anyone has a copy of this script -- it would be great to have here on the forums.

JB

[JayBaen]

Buehler ... Buehler ...

Really, I'd take it via PM, email, courrier pigeon, whatever ... :)

JB

[JayBaen]

OK .. short of beating a dead horse here ...

I've heard through the grapevine that some version of DNS replication will be built-in to the next release (point?) of IWorx ... however, I'd love to get a head start on this.

I've got a second box built (CentOS 4) replicating manually the entries from a single IWorx server (DJBDNS). I'd love the aforementioned script so that I can get multiple servers syncing to this secondary (tertiary) server(s).

Is there no-one who'd be willing to re-post the original script (broken, orphaned or whatever) -- even for archive's sake? I'd love to get this automated sooner than later.

Regards,

JB

[IWorx-Chris]

JayBean,

We will be releasing a BETA version of our DNS sync in the next (point) release. We're lookin gto release in the next 2 weeks (uh oh, I gave a release date :D) and this feature will be "hidden" so folks don't take it as non-BETA. Please contact me when the point release comes out and we'll explain how to use the sync feature.

Chris

[JayBaen]

Fabulous answer.

Thanks for the reply, I'll contact you once the release hits.

JB

[pascal]

ha ha ha chris that give a release date, incredible !!!! :cool: :p :D :eek:
;)

[timryberg]

It DOES happen once in a while ;-)

[IWorx-Chris]

haha, once in a while :D.. and to be even more specific the point release will be out Thursday. It's mainly a bugfix release but does have some oft requested goodies which we'll detail in the changelog.

Chris

[JayBaen]

... Please contact me when the point release comes out and we'll explain how to use the sync feature.

Did this hidden feature make it?

[IWorx-Chris]

It did. Anyone wishing to use the BETA DNS sync between 2 iworx-cp boxes go to (after you login to nodeworx):

https://<your server>:2443/nodeworx/dns.php?mode=sync

You can then add "listening" servers to the current iworx-cp server which will grab dns info on a 5 minute basis and publish the dns data in conjunction with any location dns info.

Chris

[LordOfLA]

This is sort of working nicely, almost... :p

I've just finished setting up 2 servers afresh with my new iworx licenses after falling in love at first sight on the 25th. I need to get a 3rd sometime soon.

I've added 2 zones and configured them on what I consider my primary dns server and set this up to publish to and listen for updates from what I consider to be my tertiary server, and vice versa.

However my "primary" doesnt seem to be publishing data to my "tertiary" however the opposite is happening. My "tertiary" is sending blank data back to my "primary".

Any ideas?

[LordOfLA]

hmm I recreated the setup and added a siteworx account and its synchronised okay.

However there is an interesting development if you look here: http://www.dnsreport.com/tools/dnsreport.ch?domain=digital-euphoria.net you'll see what I mean :p

[LordOfLA]

I've disabled synchronisation for the moment. I need my dns to work :p

I'm wondering if it wouldn't be best to use the API to push changes to a persons "primary" server and have all subsequent nameservers refresh their data from the "primary" every 15 minutes to avoid mass record duplication which leads to dns breaking.

Their also needs to be some way to keep the serials synchronised across machines too.

[IWorx-Chris]

LordOfLA,

What was your exact DNS setup with the sync tool? There should be no dup records if you're doing a one to one sync and what you describe about setting up a "primary" and having it sync every so often is how the cur system works.

Chris

[LordOfLA]

Chris,

I had my "primary" pushing updates to my third dns machine, as well as having the third push updates to my primary.

Wanting all 3 machines to have the same data if dns zones are being added on different machines I need the other 2 to get the data pushed over to them.

However the data pushed back to the primary includes what the primary already has and ends up being duplicated, rinse, repeat. It had got to the stage where tinydns just wasnt answering lookup requests due to the number of records in a potential response.

eg:

I have say ns0.digital-euphoria record on primary.
primary pushes that to tertiary which had no data but now has ns0.digital-euphoria.net record.

Tertiary sends that data back to primary. which merges in the new data and has a duplicate entry.

This gets pushed out to tertiary which merges in the information and now has 3 duplicate entries.

This gets pushed back to primary which now ends up with 5, etc....

It was amusing to watch until tinydns broke down and asked me just what it was I wanted it to do :p

[IWorx-Chris]

I have say ns0.digital-euphoria record on primary.
primary pushes that to tertiary which had no data but now has ns0.digital-euphoria.net record.

Tertiary sends that data back to primary. which merges in the new data and has a duplicate entry.

This gets pushed out to tertiary which merges in the information and now has 3 duplicate entries.

This gets pushed back to primary which now ends up with 5, etc....

It was amusing to watch until tinydns broke down and asked me just what it was I wanted it to do


Actually that's not how the system works so I'd like to check this out more if you have it setup still. It'll never combine the polled data and pass it on (i.e. it's not transitive). If box A is listening to only box B and box C is listening to only box B then box C will *never* have any of box A's info. It was designed this way for just this "cycle" problemn you mention.

Chris

[LordOfLA]

I had box A listen to and publish to box B and box b to listen to and publish to box A.

If there is a better way to make sure all 3 dns servers serve all the domains created on any of the 3 machines how would I go about it?

I can recreate the setup if you like and you can login to the boxes to watch.

I haven't gotten my third iw-cp license yet so we cant check what happens with 3 machines just yet however.

[pascal]

It did. Anyone wishing to use the BETA DNS sync between 2 iworx-cp boxes go to (after you login to nodeworx):

https://<your server>:2443/nodeworx/dns.php?mode=sync

You can then add "listening" servers to the current iworx-cp server which will grab dns info on a 5 minute basis and publish the dns data in conjunction with any location dns info.

Chris

Hi

Not sure to understand. Sorry.

I have 3 Iworx boxes.
But I have only 1 *main* dns servers located on BOX 1 (another instance of djbdns)
Today I do sync of dns data (data.cdb) from BOX2 and BOX3 to BOX1

So I have only one ns1 et ns2 servers for all my boxes (located on BOX1)

With this tool, how I should set it up ?

Enter in BOX1 then add BOX2 and BOX3 in BOX1's sync tool
or
Enter in BOX2 and BOX3 's sync tools and add BOX1 ?

I think this is the last solution, isn't it ?

Also does it sync the main interworx data.cdb (that is also sync to all IP on the same boxes) ? or do I have to define the IP of the instance of my djbdns ?

Thanks

Pascal
ps : the dns.php?mode=sync works on one of my box but not of the other (both are at 2.1.2 release level)

[LordOfLA]

Any progress with this? I'd really rather not have 3 machines pretend to be 2 dns servers if I can help it :)

[LordOfLA]

Just as an update. I emailed IWorx support yesterday and they've fixed the bug in the synchronisation module and patched my servers with it. I now have my two iworx CP servers synchronising dns data between themselves happily.

Soon I will be testing synchro between 3 servers :)

[JayBaen]

I'm going to come full-circle with this post and harken back to (what I believe to be) GetHosted's original thought - which was to sync *external* DNS servers to an (or multiple) IWorx boxes - which for some of us, there is still great benefit.

Preface: to replicate the DNS from one IWorx box to another, you should use the new DNS syncing tool provided from within IWorx. It works fabulously (as expected). Don't bother with anything else in this post.

If however, you're still interested in being able to export the DNS from one (or multiple) IWorx boxes to a standalone (non-IWorx) DNS server, but can't seem to find GetHosted's aforementioned script, or can't make it work, then this should help.

Firstly, I hope I'm not over-stepping my boundries here, but I've attached GetHosted's original (unaltered) dns_sync script. The following are the changes I made to get it functional.

--------------

* I changed the DJBDNS install to match mine (which is Bernstein's default).

$TINYDNS_DIR='/etc/tinydns'; // note lack of trailing slash

* N.B. If you're going to only sync with one external IWorx server (either for testing or intended), make sure you only have one entry in the array (and no matter how many you use, they must be referenced by DNS name, not IP). Multiple servers = multiple DNS names ... array('fooblah1.com','fooblah2.com');

$IWORX_SERVERS = array('fooblah.com'); // to add additional servers

* The real fix was adding the missing SOA record. Without this, the sync happens but I could never get any of the DNS records to actually query.

case "SOA":
$this_server_data.= "Z".$record['domain'].":".$record['nameserver'].":".$record['contact'].":".$record['serial'].":".$record['refresh'].":".$record['retry'].":".$record['expire'].":".$record['minimum']."\n";
break;

* For completeness I added the PTR record (which a bit pointless for me, but since it's available in IWorx I put it in) and the TXT record (which is nice for the SPF entries):

case "TXT":
$this_server_data.= "'".$record['domain'].":".$record['target'].":".$record['ttl']."\n";
break;
case "PTR":
$this_server_data.= "^".$record['target'].":".$record['host'].":".$record['ttl']."\n";
break;

----------------

And, voila, they sync.

Regards,

JB

[WebXtrA]

I'm sharing this, just in case somebody got the same problem as me:

The "Z" shoud be a "D" in this piece of code:
case "SOA":
$this_server_data.= "Z".$record['domain'].":".$record['nameserver'].":".$record['contact'].":".$record['serial'].":".$record['refresh'].":".$record['retry'].":".$record['expire'].":".$record['minimum']."\n";
break;

If you keep it like this, your SOA admin contact would be admin@domain@com

If you change it in this:
case "SOA":
$this_server_data.= "D".$record['domain'].":".$record['nameserver'].":".$record['contact'].":".$record['serial'].":".$record['refresh'].":".$record['retry'].":".$record['expire'].":".$record['minimum']."\n";
break;

When you have it like this, your SOA admin contact would be admin@domain.com

Well, it's important for .de and .nl TLDs, otherwise you can't register domains with your nameservers.

EDIT: That worked only for a few seconds...

I added the second line to the code to fix this issue:
case "SOA":
$record['contact'] = strtr($record['contact'], "@", ".");
$this_server_data.= "Z".$record['domain'].":".$record['nameserver'].":".$record['contact'].":".$record['serial'].":".$record['refresh'].":".$record['retry'].":".$record['expire'].":".$record['minimum']."\n";
break;

[pascal]

It did. Anyone wishing to use the BETA DNS sync between 2 iworx-cp boxes go to (after you login to nodeworx):

https://<your server>:2443/nodeworx/dns.php?mode=sync

You can then add "listening" servers to the current iworx-cp server which will grab dns info on a 5 minute basis and publish the dns data in conjunction with any location dns info.

Chris

Ok I have server A where I have my own dns servers.
I Want server B send updates DNS to server A

So on server B I did :
- In the overview DNS I have enter the 2 servers ns1.x.net and ns2.x.net that is set on the overview dns of server A (same name = same ip)
- I have setup the dns-sync tool to add as listener server A

If I understand well, all dns changes on servers B will be replicate on server A
so all my customers may use the same dns servers adersses

Is it right ?

Pascal

[johnbrand]

Hey guys,

My interworx server has crashed today. And the DNS_sync tool is working on an other server. But I think the sync tool isn't functioning the way it should when something like this happens.

It just empties the DNS list, because it can't connect with the original server. So it can't resolve any domains where it was ment for.

Can somebody help me to fix the problem so when the server blacksout it is still possible to resolve the domains.

Yours sincerely,

John

[WebXtrA]

You can create a dedicated secondary DNS server and update this DNS server with steps 7 and 8 of this howto:
http://www.interworx.com/forums/showthread.php?t=1540&highlight=DNS

When one server crashes, then the records are still there on the secondary DNS server.

[JayBaen]

@ Chris --

Do you know if it's possible to alter the remote-access.ini file, and have the changes not get overwritten for IWorx boxes located at certain Data Centers?

I ask because it seems that certain DCs overwrite this file when they sense it's been altered (may be a function of Noxworx?). If it gets overwritten, it's impossible to do any manual syncing at all, since one can't place "trusted" IPs in the remote-access.ini.

Is this true?

JB

[JayBaen]

erm ... hello?

:)

[JayBaen]

OK -- @ anyone IWorx?

JB

Buehler ... Buehler ...

[IWorx-Socheat]

Hi Jaybean,

Sorry for the delay on this. This is a bug on our part in the dns-export.php code. You are correct, it doesn't affect everybody, so if you are finding your remote-access.ini being rewritten, open a ticket and we'll get your box patched up.

Socheat

[JayBaen]

Excellent.

2 Tickets submitted (2 servers of 3 effected).

JB

[JayBaen]

Worked like a charm.

Thanks again.

JB

[kipper3d]

Hello guys/gals

Whats the word on this dns syncronization now? Interworx appears to have this built in.

I currently purchased 2 VPS with different ip blocks for primary and secondary DNS and was planning to sync all interworx servers. But after reading through this thread I've this feeling this information is a bit outdated.

If anyone has information to share on this subject I would greatly be indebted to them!

From what I can gather from the documentation and this old thread.


Use VPS Intworx cp on both vps1 and vps2
Make them masters via creating API key
Apply that key to all the other interworx servers via dns sync area
Change NS1 and NS2 to the IPs of vps1 and vps2


Seems I can only have one master, so what do I do about NS2 the primary NS?

I currently using NS1-12 as all servers are hosting their own DNS. Know how hard it is to get some people to update their NS. Can I continue to utilize all those NS from Master and then just provide new customers with NS1 and NS2?

Thank you so much in advance!

[EverythingWeb]

You say "planning to sync all interworx servers"

does this mean that vps1 & vps2 are running Interworx?

[kipper3d]

does this mean that vps1 & vps2 are running Interworx?

Not yet. I am trying to find out what I need to do first. I already jumped the gun here and am trying to see what the new process is before proceeding any further. I am not even sure I need 2 vps as it appears I can only utilize 1 master. My guess would be that theres 1 master and each server plays its own secondary.

[kipper3d]

No response for this thread. Does interworx dns sync even work?

Is interworx coming to an end? I used to get faster responses from this forum.

[EverythingWeb]

We use DNS Syncing across an bunch of interworx servers, and it works great.

we ahd got it setup before it was officially released, and an update (v3.0.0 I think) caused an API key issue, which was resolved easily/quickly and they sync every X minutes and it works flawlessly. :)

[kipper3d]

Thanks!

So heres my question - would this be feasible setup?

few interworx servers. ns2-14 pushes dns to vms
1 vms with interworx. vms uses ns1 (this vms does nothing but serve dns)

Each domain use ns1 and whatever number for their server as secondary?

Thats my confusion with this syncing. I've worked with large companies that set up bind primary and secondary and sometime a third and forth and non of the other servers handle any DNS records as they are handled by the primary and secondary servers.

I cannot find much docs on interworx dns sync system to fully understand this enough to move forward. So if someone can shed some light i would very much appreciate it.

Thank you!!