Frostbyte

I finally found time to update Aurora.

I attempted to do an installation with the release package that can be found over on xboxunity.
I originally had my Aurora folder on HDD, which I deleted prior copying the new version over. I also deleted the contents of the cache and mindex folders.

Yet for some reason ConnectX remembered my credentials, everything else had to be set up from scratch.
Can someone verify if I indeed did a fresh installation from scratch? Thanks!

Assuming that work on SMBDLL finishes (if ever), wouldn't that render connectx obsolete (since SMB will be handled natively within Aurora)? Just asking out of curiosity.

@gavin_darkglider: I don't think people are asking for an update on ConnectX, more or less they are asking for a proper (from scratch) implementation of CIFS/NFS mounting within Aurora. Does this also need to be byte-patched to work with the retail kernel?

While I do acknowledge that this might not be the most common use case, some users (including myself) have a large library of over 350 titles on a NAS device.
I'd rather be able to stream games directly from the NAS, rather than constantly freeing up space and transferring files over via FTP.

I know it's not a priority, but if and when the developers decide to bother with rewriting the plugin; my only request is make it so you can specify an IP address instead of the NETBIOS name (and possibly make credentials not a mandatory parameter).

If we're that concerned about overhead and performance, then I would suggest making an NFS plugin to go alongside the SMB one.
All the NAS devices support NFS, and there's even freeware available to set up NFS shares from Windows.

Just thought I'd give my 2 cenz again, since the ConnectX subject was brought up again (and I'm one of those who eagerly await for an optimized version).

PS:
Maybe this is a stupid conjecture, but can't we use LUA to have ConnectX only scan for the available content, and then have the selected game (upon launch) get preloaded to a temporary storage via the ftp protocol? (Then erased once we close it) ..I mean, we gotta have something (even if it's hacky) until and if we ever get a proper SMB/NFS implementation.

PS2:
Yes I know about netISO, it just doesn't fit my use case, sorry. I want to be able to browse and select directly from the console.

Fair enough.

From what I can surmise, we will get to see NOVA though.. If it's stable, it's good enough, even if NetISO doesn't make it into this release.

Should you require additional beta testers, I can help you out as well.

16 hours ago, gavin_darkglider said:

1. I am assuming you are using Samba, and not windows for your share for connectX. If this is the case, you can set it up to also run as a WINS server. If you are using windows, I dont think that non server versions include a WINS server, if those even do anymore.

2. Your welcome. As for Unity correcting the behavior, that isnt surprising, as it looks games up in the unity database by TitleID, not name. The livestrong issue makes sense also, since we have said since the beginning that some assets are pulled from live adjacent servers.

3. Copies of the logs of aurora freezing would be helpful to see what is causing this issue for you.

 

1) Correct.
2) Also correct. Exactly what I noticed.
3) Good point, I will attempt to reproduce it and supply logs.

2 hours ago, gavin_darkglider said:

1. Having VLAN support on a console doesnt make sense at all. That is something that is on switches, and routers, and if set up properly, you dont need an extra router to resolve NetBIOS. VLAN's are there to change what network ports are assigned to be able to communicate with each other and usually applies to ethernet ports on a switch, like the one in any router. You can use it with wireless, but that will put all wireless on one VLAN, as it does it on a per card basis. Which means if your WIFI card is in master mode, then all devices connected to it could be separated from the devices on the switch, or ethernet ports on the router. Might want to take a class on advanced networking, because what you are describing is wrong on a lot of different levels, just my professional opinion, since this is what I do, when I can find work. That being said, even if we were to try and add support for that to Aurora, which we probably wont, in order to use it, we would have to patch the underlying kernel in the nand to support VLAN's, which I am fairly certain it doesnt. Even running a PC with linux wont support VLAN's, unless the kernel is set up to support VLAN's.

2. You must be doing something wrong, as aurora doesnt pull names of games from the name of the folder it is in, but from the default.xex for the game. I have multiple multi disc games on my console, and have never had this issue, at least for the first disc. some game manufactures didnt put the game name into the discs default.xex after the first disc. In that case, you can either hide the extra discs with the hide multidisc filter, or you can manually rename them, using the game rename feature in aurora. Comes in handy with some games, such as Duke Nukem Forever, which still has its codename(Buckeye) in the xex.

3. This is an issue with the xbox dropping the connection to the server. This could be do to your network configuration, especially if I am understanding what you are doing, or the server load at any given time. I have mentioned on multiple occasions it would be nice if there was a CRC check implemented, especially for title updates, that way if what is downloaded doesnt match what is on the server, it will automatically erase the corrupt files, and try again. WIFI connections are the worst for this. 90% of title updates I have tried downloading while using WIFI on the xbox have been corrupted.

1) Nah, your opinion is 100% valid. I was just not thinking clearly when I wrote this. I guess the lack of sleep is to blame for this brainfart*.
IIRC you also need a WINS server if you want to resolve the NetBIOS across a different subnet.. which is also a setting not supported by the Xbox.
*I also have a tendency to over-complicate things on first try, so my apologies. I shouldn't be treating a console like a server in the first place.

2) Neat, I didn't know there was such a filter. Much appreciated.
The weird thing in this case is that when Aurora had a chance to update the assets from Unity, it corrected this behavior. (Even for Duke Nukem Forever)
One of my other findings was that you have to disable "livestrong" on dashlaunch, so other stuff like screenshots and synopsis are successfully pulled. (the download counter would drain awfully fast, so I thought something was up with that)

3) What I did notice on this, was that the download counter would get occasionally get stuck. During that time I observed no traffic on the port.
It just seems that Aurora doesn't want to time those out. And the only "safe" option, during my tests, was to shutdown and restart the Xbox.
Attempting to invoke a reboot from the GUI, launching a game, or pressing Y to reload Aurora would always freeze the Xbox.

I ended up passing the VLAN over a wireless link (trunk) and then through the 100Mbit Ethernet (access) connection of the Xbox 360.
NetBios now resolves and the Xbox is able to see the games through the NAS (via ConnectX).

That being said, I would REALLY like it, if we could have 802.1Q (VLAN) support in a future Aurora release. (So that I could possibly eliminate the extra router)
Although I realize that this might be too much of a stretch.. especially if the hardware (Wired/Wireless NIC) of the console doesn't support such functionality.

PS:
I'm a little bit unsure about the folder structure for multi-disc games.
I've read this post here, and went through with the "Situation B" of the article.
Doing this, however, causes the games to be titled "Disc 1/2/3/4" instead of their XeX title.

PS2:
This seems to be related with the connectivity to the Aurora server, as I need to retry updating assets multiple times, to fetch covers and titles.. I'm still not able to get any synopsis or screenshot info.

Ah, I didn't mean to come off as aggressive. Of course, I'm not blaming anyone for this. Something is better than nothing, right?
I'm also not questioning it's performance, I know that it would transfer faster than protocols which have overheads in them. (CIFS being the worst offender)

However, should an opportunity present itself to speak with cOz, I would love to discuss on how to bring more flexibility to this solution.

Thank you for the information so far.

2 hours ago, gavin_darkglider said:

1. CIFS/NFS share is pretty much what is going on with connectX. 

2. Extracted ISO doesnt work with NetISO. Only GOD, and ISO.

3. All network traffic is a Server/Client communication system. It doesnt matter what the server is, so being a network based thing there needs to be a server, and designing the server ourselves gives us more options on expansion, and how things are handled, which allows us to stream information stored in the iso images, that gets stored into a DB on the server side, so we can send preview images, and title information, that is stored in the iso/god file(s), that we couldnt send otherwise, and if we handled it on the xbox side, would slow down loading of the information, or would make the process less dynamic, if you were to change the files on the server. It would add loading time to the service, as the xbox would have to load all of the images, from the server, to pull the information, then arrange it in a temporary DB, to do it all over again.

4. When you mount an image with netIso service, you can browse the files in the image like you can with any disc you put in the drive, as it emulates the drive on the console, with the iso mounted to it. This will allow any copy/paste operation you can do with a disc. You will not be able to move or delete files off of an iso image, as it is a read only file system.

5. Audio/Video ISO's arent currently supported, but could easily be added to the server in the future. We need to make sure everything else is working as expected, before we worry about that, as the service is primarily for creating a faster and more stable game streaming service. We cant fix connectX, as that was written by Microsoft, and dont have the code.

6. Any nas device should be supported, once the server code is ported from windows to linux, as most Nas systems are running linux. And the current plan is to release the code for the server, so other people can port it, and make it fit the needs of more light weight devices, such as NAS systems, routers running custom firmwares, etc. Unfortunatly with any linux system, it is going to be a challenge, as you will need to compile with the same toolchain as the system on your device was compiled with. Since the GPLv2 which the linux kernel is released under, states that if you sell any device running linux, you have to not only release the code for the GPLv2 code, but also the toolchain you used to build said code, this wont be much of an issue, if you know how to tweak GCC. 

6. For reasons stated previously, we wont switch it to Ciffs/NFS. As for NFS, most of our users are running windows on their PC's anyway, which is part of the reason why the server was written for windows first and foremost. 

7. Yes, you can only have one ISO mounted per xbox at a time. This is a limitation in the xbox kernel, since the client software mounts the image at the DVD device handle. There might be a way around this, if we take advantage of the HD-DVD drive handle, but if we do, the kernel wouldnt recognize games there, just movies, as that attachment was made specifically for DVD/HD-DVD's. Not that that fad went anywhere with blueray coming out like 6 months later. lol. 

1) I don't think they've implemented it correctly, as it can only utilize the SMB protocol (CIFS) and can only connect through the NetBios name (restricting you to the same sub-net). These are two very serious limitations.

2) I see. However, there have been some reports that the GOD format is kind of slow, and ISOs have padding as we know (so they are not space friendly exactly).

3) A binary server utilizing a proprietary protocol != widespread file sharing protocols. I do agree with your point of view on the rest, but essentially you're solving the loading time problems by restricting it to one element at a time, and an element you choose from the server nonetheless. That means that you can never have a true gallery over the network, where the Xbox can dynamically choose a title from. Why store them on a temporary DB and not have a persistent cache mechanism, which would only update on the first negotiation, upon hovering-on/launching a title, or on demand via an option? Most people have gigabit on their home networks, and well.. WiFi was never advised as a good way to begin with.

4) Yes, I'm totally fine with a read-only file system, as that's how I usually do my shares for media center devices. I only mentioned the write/delete options, just so the Xbox can update the metadata on the source (NAS) without having to invoke an internet connection every time, to update the local cache.

5) Agreed. But just to be clear, I wasn't asking for a fix on ConnectX (I know it has been made by M$), I was just investigating whether it's possible to write something (from scratch) that would supersede it and overcome it's current limitations. (Source code should be available for mounting CIFS/NFS shares)
I know that I'm flirting with a response of the type "Then write your own and present it to us". Which I would be more than happy to take a shot at, if there is any kind of material I can study, so I can help developing it.

6) In theory yes, but not all NAS devices offer Shell/SSH Access options. And for those that do, they're pretty bare-bones, often times missing the required libraries to do these stuff. For example take the Synology NAS and their RADIUS package, should you require connectivity of the Daemon with a MySQL Service, you're SOL (Because whatever their web interface doesn't offer as an option, the daemon hasn't been compiled with), that means that you've got to use OptWare for these cases. And even then, this does not come remotely as close as a Debian/RHEL repository where you can download required libraries on demand. The only NAS I see, which could be easily supported, would be FreeNAS. Or if you guys make this as a standalone package with no need for additional libraries or dependencies.
These being said, I agree that the majority of the users use their PCs for this. But even this becomes a little bit impractical, when you have a library spanning over 300 titles.
I'm not against it, but I would like to see more options (in the future) regarding on how to do stuff over the network.

7) And this is what baffled me the most, the choice to emulate the DVD drive. Surely, couldn't there be a better way to do it with network share mounting? (Local cache, algorithms to efficiently decide what to update and when, option to enable/disable share(s), option to automount share(s), option to delete share(s) along with their cache, etc..)

Hello and happy new year!

First and foremost let me say a huge thank you for all the time and effort you have put into this bad boy.

However, I'm a little bit confused about this whole NetISO development choice.

From what I can surmise (correct me if I'm wrong) so far:

1. You can mount an ISO, GOD, or Folder (extracted ISO) to it.
2. You can only have one thing mounted/offered at a given time, but to multiple consoles.
3. It requires a server binary to be running on a PC. Currently only for Windows (and possibly Linux, should the binary be compiled in the future).

I would much more prefer a way to natively mount a CIFS/NFS share from a NAS/PC to the Xbox.
And to be able to perform FileSystem operations between the Share and the Xbox (Copy, Paste, Move, Delete, Execute, etc).

This would, in turn:

1. Eliminate the need for a server binary.
2. Be able to stream more than one game, at a given time.
3. Be able to copy games from a NAS/PC to the Xbox. (Case: wire speed is slow, but a centralized network repository of games is desired)
4. Be able to stream/copy/paste/move/delete other types of media, like video files. (If this will be a thing in the future)
5. Be much more flexible in terms of what kind of NAS devices can be supported.

These being said, I acknowledge that this is better than what ConnectX offers at the moment.
I have my NAS on a different VLAN/Subnet, and because ConnectX does not offer a way to utilize an IP address (only the NetBios name), I cannot use it.

Sure, I could open up NetISO on my PC and have it stream content from the NAS network path. But that would mean transmitting and receiving the same content, from a man-in-the-middle device, halving the effective bandwidth. I would much more prefer a direct access approach.

Also, if people use their PCs and are in need of more transfer speed than what a CIFS Share offers, they should opt to configure an NFS Server/Share.